-- Logs begin at Wed 2022-08-10 09:22:20 UTC, end at Wed 2022-08-10 10:20:32 UTC. -- Aug 10 09:49:32.330217 dv-cinder-850836-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 09:49:32.340943 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 09:49:32 2022] *** Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: nodename: dv-cinder-850836-5-iscsi Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: machine: x86_64 Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: clock source: unix Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: pcre jit disabled Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: detected number of CPU cores: 4 Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: current working directory: / Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: detected binary path: /usr/bin/uwsgi-core Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: your processes number limit is 47680 Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: your memory page size is 4096 bytes Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: detected max file descriptor number: 2048 Aug 10 09:49:32.343094 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: lock engine: pthread robust mutexes Aug 10 09:49:32.343823 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: thunder lock: enabled Aug 10 09:49:32.343823 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 09:49:32.343823 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 09:49:32.382113 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: Python main interpreter initialized at 0x55cbad29a9c0 Aug 10 09:49:32.382113 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: python threads support enabled Aug 10 09:49:32.382113 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: your server socket listen backlog is limited to 100 connections Aug 10 09:49:32.382113 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: your mercy for graceful operations on workers is 90 seconds Aug 10 09:49:32.382433 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 09:49:32.382506 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: *** Operational MODE: preforking *** Aug 10 09:49:32.382773 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: *** uWSGI is running in multiple interpreter mode *** Aug 10 09:49:32.382773 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: spawned uWSGI master process (pid: 80235) Aug 10 09:49:32.383053 dv-cinder-850836-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 09:49:32.383757 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: spawned uWSGI worker 1 (pid: 80246, cores: 1) Aug 10 09:49:32.383927 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: spawned uWSGI worker 2 (pid: 80247, cores: 1) Aug 10 09:49:32.384072 dv-cinder-850836-5-iscsi devstack@keystone.service[80235]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 09:49:37.868020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 09:49:37.868020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: from cryptography.utils import int_from_bytes Aug 10 09:49:37.868787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 09:49:37.868787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: from cryptography.utils import int_from_bytes Aug 10 09:49:37.868908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 09:49:37.868908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: from cryptography.utils import int_from_bytes Aug 10 09:49:37.868908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 09:49:37.868908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: from cryptography.utils import int_from_bytes Aug 10 09:49:37.918413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 09:49:37.918557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 09:49:37.918651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 09:49:37.918728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 09:49:37.918788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] command line args: [] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 09:49:37.918887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] command line args: [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 09:49:37.919069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 09:49:37.919167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 09:49:37.919234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ================================================================================ {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 09:49:37.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ================================================================================ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 09:49:37.919499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] admin_token = **** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.919585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] admin_token = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.919744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] config_dir = [] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.919835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] config_dir = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.919931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.920016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.920094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] config_source = [] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.920164 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] config_source = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.920274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] debug = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.920360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] debug = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.920470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.920736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.920832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.920892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.921026 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.921110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.921243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.921323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.921489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.921596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.921736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.921837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] list_limit = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.921939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_config_append = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] log_config_append = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_dir = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] log_dir = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] log_file = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_file = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.922942 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.923923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.924972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.925103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.925216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.925299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.925455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.925575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.925728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.925817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.926210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.926287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.926390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] publish_errors = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.926467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] publish_errors = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.926619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.926694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.926848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.926933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.927042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.927113 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.927281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.927419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.927504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.927574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.927737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] standard_threads = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.927817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] standard_threads = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.927964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] use_journal = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_journal = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_json = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] use_json = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_stderr = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.928940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] use_stderr = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.929044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_syslog = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.929133 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] use_syslog = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.929271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.929353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 09:49:37.929524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.929606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.929759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.930325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.930527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.930610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.930767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.930843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.931041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.931134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.931281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.931358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.931525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.931603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.931760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.931858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.932006 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.932138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.932253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.932472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.932551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.932722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.932801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.932958 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.933081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.933191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.933269 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.933433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.933523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.933676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.933761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.933913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.933990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.934135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.934208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.934395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.934473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.934636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.934719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.934872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.934972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.935134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.935221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.935389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.935467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.935647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.935725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.935900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.936003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.936126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.936214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.936531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.936630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.936703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.936773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.936842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.936922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.937045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.937129 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.937292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.937371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.937519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.937610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.937749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.937827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.937982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.938058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.938233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.938319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.938484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.938562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.938724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.938801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.938973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.939069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.939207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.939292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.939441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.939519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.939667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.939819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.939893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.939962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.940113 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.940198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.940343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.940429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.940572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.940649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.940811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.940895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.941044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.941130 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.941281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.941359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.941507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.941585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.941741 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.941870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.941972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.942050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.942206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.942309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.942438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.942552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.942656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.942751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.942895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.943027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.943165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.943275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.943395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.943514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.943589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.943704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.943818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.943940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.944049 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.944127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.944290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.944370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.944523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.944644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.944752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.944844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.944983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.945071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.945234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.945313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.945519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.945614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.946269 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.946349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.946506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.946594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.946742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.946828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.947023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.947150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.947263 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.947343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.947506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.947589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.947744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.947826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.947979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.948052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.948213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.948334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.948448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.948525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.948685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.948762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.948921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.949001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.949156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.949233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.949390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.949468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.949625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.949705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.949860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.949937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.950092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.950169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.950326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.950406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.950559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.950637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.950811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.950895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.951076 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.951156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.951310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.951392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.951551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.951630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.951776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.951849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.952007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.952086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.952242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.952320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.952470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.952559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.952697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.952776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.952924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.953002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.953152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.953229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.953377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.953456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.953604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.953682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.953838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.953924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.954071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.954150 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.954309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.954386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.954547 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.954632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.954806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.954890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.955072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.955153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.955305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] auth.external = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.955396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.955535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.955626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.external = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.955765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.955842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.956043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.956132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.956272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] auth.password = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.956361 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.956501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.password = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.956579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] auth.token = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.956733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.token = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.956812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.956967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.957054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.957200 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.957278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.957432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.957509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.957665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.957750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.957913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.957992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.958140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.958220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.958375 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.958461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] credential.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.958615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.958722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.958861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.958963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.959131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.959227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.959367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.959448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.959603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.959681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.959833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.959919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.960070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.960149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.960333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.960419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.960568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.960655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.960810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.960888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.961086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.961263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.961371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.961519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.961623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.961785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.961888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.962030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.962131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.962288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.962389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.962524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] federation.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.962625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.962771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.962850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.963041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.963125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.963290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.963379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.963521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.963606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.963767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.963854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.964014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.964093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.964266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.964346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.964503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.964583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.964753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.964830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.964987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.965077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.965231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.965310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.965464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.965542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.965709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.965862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.965937 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.966007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.966176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.966256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.966514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.966644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.966722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.966793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.966929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.967024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.967189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.967281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.967439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.967518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.967695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.967774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.967929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.968016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.968161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.968241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.968389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.968467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.968632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.968723 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.968876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.968997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.969123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.969202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.969373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.969460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.969617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.969697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.969886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.969965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.970123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.970214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.970359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.970436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.970588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.970665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.970823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.970910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.971078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.971159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.971316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.971395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.971553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.971643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.971790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.971869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.972034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.972123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.972264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.972351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.972508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.972635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.972753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.972834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.972988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.973075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.973230 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.973309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.973474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.973554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.973702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.973792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.973938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.974017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.974166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.974244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.974399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.974486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.974635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.974713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.974872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.974982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.975131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.975213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.975367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.975435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.975612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.975679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.977403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.977527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.977527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.977527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.977527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.977527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.977957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.978943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.979043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.979207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.979290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.979446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.979528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.979695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.979775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.979940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.980021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.980178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.980259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.980409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.980490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.980644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.980725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.980879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.980959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.981131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.981210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.981362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.981442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.981598 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.981676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.981837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.981916 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.982077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.982206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.982310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.982389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.982553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.982632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.982781 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.982863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.983049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.983133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.983285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.983367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.983520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.983601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.983752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.983832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.983986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.984065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.984223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.984303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.984463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.984543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.984696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.984774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.984931 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.985071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.985148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.985214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.985393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.985473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.985626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.985707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.985859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.985939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.986094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.986174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.986338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.986417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.986576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.986656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.986804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.986883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.987062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.987145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.987302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] resource.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.987382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.987542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.987621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.987777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.987858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.988008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.988086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.988247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.988325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.988493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.988572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.988728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.988808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.988966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.989044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.989200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.989279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.989436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.989515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.989669 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] role.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.989749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.989917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.989997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] role.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.990141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.990220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.990381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.990459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.990616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] role.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.990698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.990861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.990939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] role.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.991114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.991466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.991628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.991709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.991870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.991949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.992114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.992210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.992895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.992983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.993130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.993210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.993377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.993466 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.993623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.993702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.993868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.993947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.994098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.994185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.994346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.994502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.994582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.994740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.994819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.995008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.995094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.995374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.995483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.995626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.995725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.995877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.995980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.996113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.996234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.996365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.996484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.996615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.996736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.996853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.996973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.997084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.997314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.997393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.997545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.997625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.997796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.997875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.998044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.998125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.998287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.998367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.998525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.998628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.998760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.998838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.999011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.999092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.999249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] token.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.999328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.999483 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.999564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.999729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.999808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:37.999959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.000037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.000198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.000280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.000437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.000517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.000672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.000750 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.000912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.000992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.001151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.001231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.001397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.001477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.001633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.001713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.001869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.001947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.002102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.002181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.002346 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.002417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.002583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.002661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.002808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.002886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.003073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.003157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.003366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.003447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.003614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.003692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.connection = **** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.003847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.003939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.004092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.004173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.004328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.004407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.004572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.004661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.004809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.004901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.005046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.005126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.005280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.005360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.005515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.005603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.005759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.005838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.005994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.006074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.006228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.006315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.006462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.006541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.006705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.006783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.006937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.007044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.007202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.007283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.007431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.007511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.007671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.007759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.007922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.008509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.008604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80246) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 09:49:38.008683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.008901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.009141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.009379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 09:49:38.009549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 09:49:38.037984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.038208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.038430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.038675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.038906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.039169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.048084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.048970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.076215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.076442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.077413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.077644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.129463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.129596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.179631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.180404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.180496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.180650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.244730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.244950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.245172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.245256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.245392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.245491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.245608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.245691 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.245818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.245901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.246030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.246244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.269295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.269541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.269788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.269922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.270069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.270202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.270333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.270462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.270589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.270728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.270872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.271017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.271017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.271256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.282165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.282499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.295490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.295721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.295856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.295985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.296135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.296262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.296389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.296516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.296642 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.296776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.296905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.297032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.297158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.297158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80246) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 09:49:38.314174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.314384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.314517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 09:49:38.314662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 09:49:38.314788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.314914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 09:49:38.315084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.315209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 09:49:38.315334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.315458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.315582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.315721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.315861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.315986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.316111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.316235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.327167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.327320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.327449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.327575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.327702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.327828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.327961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.328089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.334833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.334990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.338432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.338715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.339021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.339238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.339366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.339527 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.339655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.339795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.339937 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.340066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.340196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.340326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.340464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.340594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.340723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.340852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.340981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.341116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.341247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.341383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.341637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.341905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.364181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.365222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.365654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.365940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.366977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.367286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.370774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.371081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.371362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.371875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.372157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.372444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.377846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.378129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.378967 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.379261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.382248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.382498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.382778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.383110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.383373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.383502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.383644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.383774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.383902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.384033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.384162 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.384297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.384489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.384794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.385072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.385341 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.400424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.400703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.401642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.401925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.404684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.405880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.407824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.408122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.409031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.409328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.412466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.413718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.416174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.417457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.419308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.419608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.419890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.420181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.420457 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.420638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.420721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.420942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.421232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.421518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.421798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.422095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.437031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.437315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.437579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.437835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.438243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.438541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.438808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.439096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.444745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.446255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.446422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.446585 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.447846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.448125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.449394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.449679 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.449966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.450253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.450526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.450911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.451220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.451514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.451806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.452095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.462326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.462607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.462885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.463187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.463464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.463734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.463954 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.464044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.464240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.464525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.464812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.465095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.465371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.465646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.480910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.482719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.483861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.485705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.486757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.488340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.488657 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.489809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.490095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.490222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.490371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.492016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.492328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.492624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.497036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.498646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.499007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.500605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.501561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.501855 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.502153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.502431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.503568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.503860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.504144 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.504422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.510475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.510762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.511061 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.512416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.512712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.512994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.517830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80246) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.518111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.518388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.518663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.518932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.519236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.519516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.519790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.520161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 09:49:38.520256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.520440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.520521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.520710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.520790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.520970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.521053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.521244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.521334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.521502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80246) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.521585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.521774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.522039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.522435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.522724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.523031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.523301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.524102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.524368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 09:49:38.555889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cbad29a9c0 pid: 80246 (default app) Aug 10 09:49:38.558349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cbad29a9c0 pid: 80247 (default app) Aug 10 09:49:38.559146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d3db-c6f0-46fa-bacf-ed50a6ea92ed None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:38.559436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d3db-c6f0-46fa-bacf-ed50a6ea92ed None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:38.559705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d3db-c6f0-46fa-bacf-ed50a6ea92ed None None] PATH_INFO: `/v3` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:38.563948 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Wed Aug 10 09:49:38 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:49:46.163663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b0527-4d2c-4adc-9995-d7b81c4f8c6c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:46.164226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b0527-4d2c-4adc-9995-d7b81c4f8c6c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:46.164226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b0527-4d2c-4adc-9995-d7b81c4f8c6c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:46.168578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:49:46.173813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:46.174249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:46.174340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:46.245825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: INFO dbcounter [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] Registered counter for database keystone Aug 10 09:49:46.264459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG oslo_db.sqlalchemy.engines [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80246) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 09:49:46.270938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writer thread running {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 09:49:46.452316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG passlib.handlers.bcrypt [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80246) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 09:49:46.452664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG passlib.handlers.bcrypt [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80246) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 09:49:46.506742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:46.508370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:46.635375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 10 09:49:46.775435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b0527-4d2c-4adc-9995-d7b81c4f8c6c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:46.844063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: INFO dbcounter [None req-bc0b0527-4d2c-4adc-9995-d7b81c4f8c6c None None] Registered counter for database keystone Aug 10 09:49:46.865218 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG oslo_db.sqlalchemy.engines [None req-bc0b0527-4d2c-4adc-9995-d7b81c4f8c6c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80247) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 09:49:46.871530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writer thread running {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 09:49:46.998068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b0527-4d2c-4adc-9995-d7b81c4f8c6c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:47.002241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d4435-93c7-4b5f-835f-d2467ba640e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:47.002853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d4435-93c7-4b5f-835f-d2467ba640e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:47.003157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d4435-93c7-4b5f-835f-d2467ba640e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:47.003426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d4435-93c7-4b5f-835f-d2467ba640e0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:47.011822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d4435-93c7-4b5f-835f-d2467ba640e0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:47.062648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WARNING keystone.common.rbac_enforcer.enforcer [None req-262d4435-93c7-4b5f-835f-d2467ba640e0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 09:49:47.063003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d4435-93c7-4b5f-835f-d2467ba640e0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:47.072738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Aug 10 09:49:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 09:49:47.078715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:47.093380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971a96f-351d-4cb5-9738-0116bfd2ec46 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:47.097716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23428395-cff4-43c1-8a91-cce20cf5066f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:47.098971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428395-cff4-43c1-8a91-cce20cf5066f admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:47.099248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428395-cff4-43c1-8a91-cce20cf5066f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:47.099541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428395-cff4-43c1-8a91-cce20cf5066f admin admin] PATH_INFO: `/v3/projects/9cee49555c8143ed868a4be75b65ed07` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:47.101161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23428395-cff4-43c1-8a91-cce20cf5066f admin admin] RBAC: Authorizing `identity:get_project(project_id=9cee49555c8143ed868a4be75b65ed07)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:47.152530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WARNING keystone.common.rbac_enforcer.enforcer [None req-23428395-cff4-43c1-8a91-cce20cf5066f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 09:49:47.152867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23428395-cff4-43c1-8a91-cce20cf5066f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:47.157400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Aug 10 09:49:47 2022] GET /identity/v3/projects/9cee49555c8143ed868a4be75b65ed07 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:49:49.129655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40415c4a-4c7b-4d88-884b-dfa13b9ee178 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:49.130191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40415c4a-4c7b-4d88-884b-dfa13b9ee178 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:49.130191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40415c4a-4c7b-4d88-884b-dfa13b9ee178 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:49.134252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:49:49.139468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebaddd-f013-4a23-9369-6f2d04e7d51e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:49.139835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebaddd-f013-4a23-9369-6f2d04e7d51e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:49.140010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebaddd-f013-4a23-9369-6f2d04e7d51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:49.161312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-baebaddd-f013-4a23-9369-6f2d04e7d51e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:49.162783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-baebaddd-f013-4a23-9369-6f2d04e7d51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:49.171067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 09:49:49.192523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40415c4a-4c7b-4d88-884b-dfa13b9ee178 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:49.204813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40415c4a-4c7b-4d88-884b-dfa13b9ee178 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:49.209001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de520a38-daee-4260-9010-54850645f7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:49.209587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de520a38-daee-4260-9010-54850645f7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:49.209866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de520a38-daee-4260-9010-54850645f7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:49.210140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de520a38-daee-4260-9010-54850645f7a9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:49.218248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de520a38-daee-4260-9010-54850645f7a9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:49.220042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de520a38-daee-4260-9010-54850645f7a9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:49.228621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Aug 10 09:49:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 09:49:49.233748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebaddd-f013-4a23-9369-6f2d04e7d51e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:49.242755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebaddd-f013-4a23-9369-6f2d04e7d51e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:49.247141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd7f90-7a24-49cf-891d-0f0aff8bb775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:49.247752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd7f90-7a24-49cf-891d-0f0aff8bb775 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:49.248029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd7f90-7a24-49cf-891d-0f0aff8bb775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:49.248299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd7f90-7a24-49cf-891d-0f0aff8bb775 admin admin] PATH_INFO: `/v3/users/2ff7daf364734feb8aab110b9b2ef63b` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:49.249593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd7f90-7a24-49cf-891d-0f0aff8bb775 admin admin] RBAC: Authorizing `identity:get_user(user_id=2ff7daf364734feb8aab110b9b2ef63b)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:49.251434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd7f90-7a24-49cf-891d-0f0aff8bb775 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:49.252916 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Wed Aug 10 09:49:49 2022] GET /identity/v3/users/2ff7daf364734feb8aab110b9b2ef63b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:49:51.297800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce44e7-b059-466f-b21d-da4e9e371e30 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:51.298353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce44e7-b059-466f-b21d-da4e9e371e30 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:51.298353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce44e7-b059-466f-b21d-da4e9e371e30 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:51.299881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:49:51.304869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1598bb-cd1f-451f-b6aa-195fda178a0f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:51.306027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1598bb-cd1f-451f-b6aa-195fda178a0f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:51.306314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1598bb-cd1f-451f-b6aa-195fda178a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:51.306890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7005d69-4327-4395-90f4-95dad4e7d493 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:51.307204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7005d69-4327-4395-90f4-95dad4e7d493 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:51.307475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7005d69-4327-4395-90f4-95dad4e7d493 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:51.308666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:49:51.314347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53914f-99f8-4e1b-80cd-8cbd7ef0c58f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:51.314347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53914f-99f8-4e1b-80cd-8cbd7ef0c58f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:51.314347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53914f-99f8-4e1b-80cd-8cbd7ef0c58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:51.324799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ed1598bb-cd1f-451f-b6aa-195fda178a0f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:51.325967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ed1598bb-cd1f-451f-b6aa-195fda178a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:51.356373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG passlib.handlers.bcrypt [None req-1f53914f-99f8-4e1b-80cd-8cbd7ef0c58f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80247) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 09:49:51.356803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG passlib.handlers.bcrypt [None req-1f53914f-99f8-4e1b-80cd-8cbd7ef0c58f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80247) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 09:49:51.402822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1f53914f-99f8-4e1b-80cd-8cbd7ef0c58f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:51.403909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:49:51.404467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1f53914f-99f8-4e1b-80cd-8cbd7ef0c58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:51.426027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1598bb-cd1f-451f-b6aa-195fda178a0f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:51.449905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:51.503857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:49:51.525108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53914f-99f8-4e1b-80cd-8cbd7ef0c58f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:51.531074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1598bb-cd1f-451f-b6aa-195fda178a0f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:51.565177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb9d60-028c-4aec-94da-2fc4eb511945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:51.565743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb9d60-028c-4aec-94da-2fc4eb511945 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:51.566026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb9d60-028c-4aec-94da-2fc4eb511945 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:51.566302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb9d60-028c-4aec-94da-2fc4eb511945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:51.577328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb9d60-028c-4aec-94da-2fc4eb511945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-32bb9d60-028c-4aec-94da-2fc4eb511945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:49:51.586010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:49:51.587037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:49:51.588048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:49:51.591254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:49:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:49:51.593036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb9d60-028c-4aec-94da-2fc4eb511945 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:51.625432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53914f-99f8-4e1b-80cd-8cbd7ef0c58f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:51.627733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:51.662397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be981c-9809-458d-a036-6e5f803af20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:51.663290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be981c-9809-458d-a036-6e5f803af20e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:51.663577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be981c-9809-458d-a036-6e5f803af20e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:51.663854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be981c-9809-458d-a036-6e5f803af20e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:51.672757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:51.691505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb9d60-028c-4aec-94da-2fc4eb511945 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:51.723673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af191f6-bb0a-48e5-8de4-aa2ee5ade889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:51.724370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af191f6-bb0a-48e5-8de4-aa2ee5ade889 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:51.724547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af191f6-bb0a-48e5-8de4-aa2ee5ade889 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:51.724823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af191f6-bb0a-48e5-8de4-aa2ee5ade889 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:51.725492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af191f6-bb0a-48e5-8de4-aa2ee5ade889 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:49:51.725794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af191f6-bb0a-48e5-8de4-aa2ee5ade889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:51.727729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af191f6-bb0a-48e5-8de4-aa2ee5ade889 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:51.731712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be981c-9809-458d-a036-6e5f803af20e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:51.733237 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be981c-9809-458d-a036-6e5f803af20e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:51.740058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:49:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:51.746327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af191f6-bb0a-48e5-8de4-aa2ee5ade889 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:51.831977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 09:49:51.838549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be981c-9809-458d-a036-6e5f803af20e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:51.842805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:51.942961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be981c-9809-458d-a036-6e5f803af20e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:51.948306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af191f6-bb0a-48e5-8de4-aa2ee5ade889 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:51.975498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a1e34-ac3e-4978-9cdc-c6a93c6c36ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:51.976336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a1e34-ac3e-4978-9cdc-c6a93c6c36ff None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:51.976614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a1e34-ac3e-4978-9cdc-c6a93c6c36ff None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:51.976881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a1e34-ac3e-4978-9cdc-c6a93c6c36ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:51.981732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af22fc7-54c3-4a4a-ba90-e83ef996470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:51.982352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af22fc7-54c3-4a4a-ba90-e83ef996470e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:51.982625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af22fc7-54c3-4a4a-ba90-e83ef996470e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:51.982896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af22fc7-54c3-4a4a-ba90-e83ef996470e None admin] PATH_INFO: `/v3/users/2ff7daf364734feb8aab110b9b2ef63b` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:51.984029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af22fc7-54c3-4a4a-ba90-e83ef996470e None admin] RBAC: Authorizing `identity:get_user(user_id=2ff7daf364734feb8aab110b9b2ef63b)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:51.985473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af22fc7-54c3-4a4a-ba90-e83ef996470e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:51.987036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:49:51 2022] GET /identity/v3/users/2ff7daf364734feb8aab110b9b2ef63b => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:51.992630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af22fc7-54c3-4a4a-ba90-e83ef996470e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:51.995715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a1e34-ac3e-4978-9cdc-c6a93c6c36ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:51.997176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a1e34-ac3e-4978-9cdc-c6a93c6c36ff None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:52.006847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9c1a1e34-ac3e-4978-9cdc-c6a93c6c36ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:49:52.016137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:49:52.017010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:49:52.017795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:49:52.017795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:49:52.017795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:49:52.017795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:49:52.023059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a1e34-ac3e-4978-9cdc-c6a93c6c36ff None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:52.078284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:52.092746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af22fc7-54c3-4a4a-ba90-e83ef996470e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:52.121629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a1e34-ac3e-4978-9cdc-c6a93c6c36ff None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:52.126556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ff2c7-2c7a-43e1-8df7-0e3841cf6791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:52.127212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ff2c7-2c7a-43e1-8df7-0e3841cf6791 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:52.127492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ff2c7-2c7a-43e1-8df7-0e3841cf6791 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:52.127764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ff2c7-2c7a-43e1-8df7-0e3841cf6791 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:52.128877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ff2c7-2c7a-43e1-8df7-0e3841cf6791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:52.130306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ff2c7-2c7a-43e1-8df7-0e3841cf6791 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:52.131917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:49:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:52.137600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ff2c7-2c7a-43e1-8df7-0e3841cf6791 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:52.155240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7c0d7-711d-48e5-aa17-913a1aab3a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:52.155847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7c0d7-711d-48e5-aa17-913a1aab3a6a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:52.156126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7c0d7-711d-48e5-aa17-913a1aab3a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:52.156399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7c0d7-711d-48e5-aa17-913a1aab3a6a None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:52.157036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7c0d7-711d-48e5-aa17-913a1aab3a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:49:52.157334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7c0d7-711d-48e5-aa17-913a1aab3a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:52.158725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7c0d7-711d-48e5-aa17-913a1aab3a6a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:52.176365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:52.219711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:52.240556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ff2c7-2c7a-43e1-8df7-0e3841cf6791 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:52.276110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc6b27-3851-464f-9fd1-23ffa52a3d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:52.276728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc6b27-3851-464f-9fd1-23ffa52a3d02 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:52.277003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc6b27-3851-464f-9fd1-23ffa52a3d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:52.277272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc6b27-3851-464f-9fd1-23ffa52a3d02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:52.278000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc6b27-3851-464f-9fd1-23ffa52a3d02 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.domain.id=default, user.id=2ff7daf364734feb8aab110b9b2ef63b) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:49:52.278303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc6b27-3851-464f-9fd1-23ffa52a3d02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:52.279789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc6b27-3851-464f-9fd1-23ffa52a3d02 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:52.287092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Aug 10 09:49:52 2022] GET /identity/v3/role_assignments?user.id=2ff7daf364734feb8aab110b9b2ef63b&scope.domain.id=default&role.id=8417343e9ef64991bcf51bb9692da102 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:54.589153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adad895-5ef7-45cd-9722-77ee43077296 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:54.590174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adad895-5ef7-45cd-9722-77ee43077296 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:54.590650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adad895-5ef7-45cd-9722-77ee43077296 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:54.592249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:49:54.597666 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459cfbf-38cc-4fe1-b018-81f1ba3a447b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:54.598450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459cfbf-38cc-4fe1-b018-81f1ba3a447b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:54.598931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459cfbf-38cc-4fe1-b018-81f1ba3a447b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:54.622922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f459cfbf-38cc-4fe1-b018-81f1ba3a447b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:54.624455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f459cfbf-38cc-4fe1-b018-81f1ba3a447b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:54.704990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edda64-c861-4085-91c3-955f2dbc3b05 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:54.705272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edda64-c861-4085-91c3-955f2dbc3b05 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:54.705474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edda64-c861-4085-91c3-955f2dbc3b05 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:54.706810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 12/23] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:49:54.712331 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 12/24] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:49:54.728817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7757-fc4d-4b84-9bf6-2230d10b5d15 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:54.729410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7757-fc4d-4b84-9bf6-2230d10b5d15 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:54.729890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7757-fc4d-4b84-9bf6-2230d10b5d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:54.738696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459cfbf-38cc-4fe1-b018-81f1ba3a447b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:54.768612 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d44f7757-fc4d-4b84-9bf6-2230d10b5d15 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:54.769822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d44f7757-fc4d-4b84-9bf6-2230d10b5d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:54.831308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:54.885708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459cfbf-38cc-4fe1-b018-81f1ba3a447b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:54.917801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:49:54.919476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcf871-3453-424f-ac1b-c304bdf22f7a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:54.919958 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcf871-3453-424f-ac1b-c304bdf22f7a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:54.920441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcf871-3453-424f-ac1b-c304bdf22f7a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:54.921870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:49:54.930479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78af5f3-734e-4b8a-91f0-91e0c0961c54 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:54.931871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78af5f3-734e-4b8a-91f0-91e0c0961c54 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:54.932501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78af5f3-734e-4b8a-91f0-91e0c0961c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:54.966641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e78af5f3-734e-4b8a-91f0-91e0c0961c54 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:54.967774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e78af5f3-734e-4b8a-91f0-91e0c0961c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:54.979866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a7e6a-24eb-4b20-b42c-8ff6a9ae1651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:54.980488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a7e6a-24eb-4b20-b42c-8ff6a9ae1651 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:54.980771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a7e6a-24eb-4b20-b42c-8ff6a9ae1651 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:54.981046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a7e6a-24eb-4b20-b42c-8ff6a9ae1651 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:54.981710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a7e6a-24eb-4b20-b42c-8ff6a9ae1651 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:54.983140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a7e6a-24eb-4b20-b42c-8ff6a9ae1651 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:55.015120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:55.042175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 15/27] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:49:55.042879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78af5f3-734e-4b8a-91f0-91e0c0961c54 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:55.119538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 13/28] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 10 09:49:54 2022] POST /identity/v3/roles => generated 219 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:49:55.120174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea16dc-c2ac-4550-b77a-ff0509fe77c1 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.120474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea16dc-c2ac-4550-b77a-ff0509fe77c1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.120752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea16dc-c2ac-4550-b77a-ff0509fe77c1 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.122394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 14/29] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:49:55.127693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea16dc-c2ac-4550-b77a-ff0509fe77c1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:55.250692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea16dc-c2ac-4550-b77a-ff0509fe77c1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:55.292121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2d99a-50aa-4b7b-bedd-90f2f574c1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:55.292857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2d99a-50aa-4b7b-bedd-90f2f574c1ec None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.293154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2d99a-50aa-4b7b-bedd-90f2f574c1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.293429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2d99a-50aa-4b7b-bedd-90f2f574c1ec None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.294105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2d99a-50aa-4b7b-bedd-90f2f574c1ec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:55.295685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2d99a-50aa-4b7b-bedd-90f2f574c1ec None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:55.299450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:55.318255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 15/30] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 10 09:49:55 2022] POST /identity/v3/roles => generated 225 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:49:55.319613 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec7ab-0fc0-4c26-9aff-916012e0e75f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.320288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec7ab-0fc0-4c26-9aff-916012e0e75f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.321217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec7ab-0fc0-4c26-9aff-916012e0e75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.347089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e7fec7ab-0fc0-4c26-9aff-916012e0e75f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:55.349665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e7fec7ab-0fc0-4c26-9aff-916012e0e75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:55.373608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78af5f3-734e-4b8a-91f0-91e0c0961c54 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:55.444604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac9772-316f-4182-957c-28014d5f38cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:55.445253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac9772-316f-4182-957c-28014d5f38cf None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.445531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac9772-316f-4182-957c-28014d5f38cf None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.446342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac9772-316f-4182-957c-28014d5f38cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.467412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac9772-316f-4182-957c-28014d5f38cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:55.468917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac9772-316f-4182-957c-28014d5f38cf None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:55.485994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 16/31] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-52ac9772-316f-4182-957c-28014d5f38cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:49:55.501829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:49:55.502692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:49:55.503511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:49:55.503511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:49:55.503511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:49:55.503511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 16/32] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:49:55.510664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec7ab-0fc0-4c26-9aff-916012e0e75f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:55.515015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac9772-316f-4182-957c-28014d5f38cf None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:55.556009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:55.656090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac9772-316f-4182-957c-28014d5f38cf None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:55.664451 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec7ab-0fc0-4c26-9aff-916012e0e75f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:55.695460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806678ed-7da7-4be8-a0cb-002f5d53c5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:55.696731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806678ed-7da7-4be8-a0cb-002f5d53c5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.697011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806678ed-7da7-4be8-a0cb-002f5d53c5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.697284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806678ed-7da7-4be8-a0cb-002f5d53c5b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.698213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:55.715923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d5279-061a-4167-b782-ca7e53ea81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:55.716692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d5279-061a-4167-b782-ca7e53ea81f6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.717980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d5279-061a-4167-b782-ca7e53ea81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.717980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d5279-061a-4167-b782-ca7e53ea81f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.717980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d5279-061a-4167-b782-ca7e53ea81f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:49:55.718507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d5279-061a-4167-b782-ca7e53ea81f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:55.719774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806678ed-7da7-4be8-a0cb-002f5d53c5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:49:55.720365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d5279-061a-4167-b782-ca7e53ea81f6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-806678ed-7da7-4be8-a0cb-002f5d53c5b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:49:55.727017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:49:55.727840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:49:55.728601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:49:55.730745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 17/33] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:49:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:49:55.738633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806678ed-7da7-4be8-a0cb-002f5d53c5b0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:55.739711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 17/34] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:49:55.746260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d5279-061a-4167-b782-ca7e53ea81f6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:55.852876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:55.867011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806678ed-7da7-4be8-a0cb-002f5d53c5b0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:55.878583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d5279-061a-4167-b782-ca7e53ea81f6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:55.910614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa950bf-f343-443e-bc32-240e93bd4019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:55.911674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa950bf-f343-443e-bc32-240e93bd4019 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.912052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa950bf-f343-443e-bc32-240e93bd4019 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.912421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa950bf-f343-443e-bc32-240e93bd4019 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.913463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa950bf-f343-443e-bc32-240e93bd4019 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:49:55.914284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa950bf-f343-443e-bc32-240e93bd4019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:55.916787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa950bf-f343-443e-bc32-240e93bd4019 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:55.919768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265e312-3ef8-4396-aa96-2510a81b3865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:55.920602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265e312-3ef8-4396-aa96-2510a81b3865 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.921381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265e312-3ef8-4396-aa96-2510a81b3865 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.921798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265e312-3ef8-4396-aa96-2510a81b3865 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.922578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265e312-3ef8-4396-aa96-2510a81b3865 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:55.924187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265e312-3ef8-4396-aa96-2510a81b3865 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:55.931342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 18/35] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:49:55.932602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878725a-2cda-4331-9ff2-f21501ba3dd8 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.932844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878725a-2cda-4331-9ff2-f21501ba3dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.933111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878725a-2cda-4331-9ff2-f21501ba3dd8 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.934466 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 19/36] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:49:55.937511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878725a-2cda-4331-9ff2-f21501ba3dd8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:55.964190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 18/37] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:49:55 2022] POST /identity/v3/projects => generated 301 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:49:55.964190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1728-8422-4fec-89c9-9538d310eed2 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:55.964190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1728-8422-4fec-89c9-9538d310eed2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:55.964190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1728-8422-4fec-89c9-9538d310eed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:55.981088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ee3e1728-8422-4fec-89c9-9538d310eed2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:55.982509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ee3e1728-8422-4fec-89c9-9538d310eed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:56.014867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:56.060158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 19/38] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:49:56.061066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878725a-2cda-4331-9ff2-f21501ba3dd8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:56.083452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e1728-8422-4fec-89c9-9538d310eed2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:56.112253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b2bd3-d1bf-4ec2-a0c6-4d971d296950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:56.112822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b2bd3-d1bf-4ec2-a0c6-4d971d296950 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:56.113377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b2bd3-d1bf-4ec2-a0c6-4d971d296950 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:56.113653 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b2bd3-d1bf-4ec2-a0c6-4d971d296950 None admin] PATH_INFO: `/v3/users/2ff7daf364734feb8aab110b9b2ef63b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:56.116081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b2bd3-d1bf-4ec2-a0c6-4d971d296950 None admin] RBAC: Authorizing `identity:get_user(user_id=2ff7daf364734feb8aab110b9b2ef63b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:56.118159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b2bd3-d1bf-4ec2-a0c6-4d971d296950 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:56.120992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 20/39] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:49:55 2022] GET /identity/v3/users/2ff7daf364734feb8aab110b9b2ef63b => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:56.129015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b2bd3-d1bf-4ec2-a0c6-4d971d296950 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:56.154761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:56.220738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e1728-8422-4fec-89c9-9538d310eed2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:56.251418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b2bd3-d1bf-4ec2-a0c6-4d971d296950 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:56.257309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aebe5-699f-407a-9f71-e44e7931171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:56.257918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aebe5-699f-407a-9f71-e44e7931171d None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:56.258197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aebe5-699f-407a-9f71-e44e7931171d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:56.259046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aebe5-699f-407a-9f71-e44e7931171d None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:56.259681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4aebe5-699f-407a-9f71-e44e7931171d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:56.261069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4aebe5-699f-407a-9f71-e44e7931171d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:56.276967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 20/40] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 10 09:49:56 2022] POST /identity/v3/roles => generated 223 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:49:56.299852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49579dc0-0b37-4ef1-95da-645a02c30281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:56.300513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49579dc0-0b37-4ef1-95da-645a02c30281 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:56.300797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49579dc0-0b37-4ef1-95da-645a02c30281 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:56.301077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49579dc0-0b37-4ef1-95da-645a02c30281 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:56.302293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49579dc0-0b37-4ef1-95da-645a02c30281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:56.303818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49579dc0-0b37-4ef1-95da-645a02c30281 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:56.306318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 21/41] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:49:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:56.312219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aebe5-699f-407a-9f71-e44e7931171d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:56.333250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029e704-bbef-40d6-b3cd-371ffaff2b24 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:56.333685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029e704-bbef-40d6-b3cd-371ffaff2b24 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:56.334059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029e704-bbef-40d6-b3cd-371ffaff2b24 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:56.335759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 22/42] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:49:56.342946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b177ac-d006-4e87-b4a7-8753a72941bc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:56.343411 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b177ac-d006-4e87-b4a7-8753a72941bc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:56.343756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b177ac-d006-4e87-b4a7-8753a72941bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:56.346767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:56.363853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-16b177ac-d006-4e87-b4a7-8753a72941bc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:56.365101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-16b177ac-d006-4e87-b4a7-8753a72941bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:56.435395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aebe5-699f-407a-9f71-e44e7931171d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:56.445343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 23/43] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:49:56.468261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b177ac-d006-4e87-b4a7-8753a72941bc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:56.471823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5c2a2-948f-4f81-a568-4f48f1d1b997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:56.472654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5c2a2-948f-4f81-a568-4f48f1d1b997 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:56.472947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5c2a2-948f-4f81-a568-4f48f1d1b997 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:56.473361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5c2a2-948f-4f81-a568-4f48f1d1b997 None admin] PATH_INFO: `/v3/domains/default/users/2ff7daf364734feb8aab110b9b2ef63b/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:56.475377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5c2a2-948f-4f81-a568-4f48f1d1b997 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2ff7daf364734feb8aab110b9b2ef63b, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:56.476849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5c2a2-948f-4f81-a568-4f48f1d1b997 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:56.487874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:56.494456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 21/44] 10.222.0.20 () {58 vars in 1361 bytes} [Wed Aug 10 09:49:56 2022] PUT /identity/v3/domains/default/users/2ff7daf364734feb8aab110b9b2ef63b/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:49:56.580937 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b177ac-d006-4e87-b4a7-8753a72941bc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:56.615740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed72899-cf62-40e6-80e2-72a7de9d7a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:56.616300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed72899-cf62-40e6-80e2-72a7de9d7a4b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:56.616583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed72899-cf62-40e6-80e2-72a7de9d7a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:56.617110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed72899-cf62-40e6-80e2-72a7de9d7a4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:56.618796 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed72899-cf62-40e6-80e2-72a7de9d7a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:56.620311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed72899-cf62-40e6-80e2-72a7de9d7a4b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:56.621938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 24/45] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:49:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:56.627961 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5c2a2-948f-4f81-a568-4f48f1d1b997 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:56.713823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:56.729730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5c2a2-948f-4f81-a568-4f48f1d1b997 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:56.762742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862ca4c-7fff-44c5-b2f9-00f48ff3eb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:56.763383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862ca4c-7fff-44c5-b2f9-00f48ff3eb40 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:56.763663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862ca4c-7fff-44c5-b2f9-00f48ff3eb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:56.763934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862ca4c-7fff-44c5-b2f9-00f48ff3eb40 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:56.764600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862ca4c-7fff-44c5-b2f9-00f48ff3eb40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:56.766048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862ca4c-7fff-44c5-b2f9-00f48ff3eb40 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:56.793325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:49:56 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:49:58.463726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9d793-ef61-41e2-b6fc-cdf53f7ae488 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:58.464249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9d793-ef61-41e2-b6fc-cdf53f7ae488 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:58.464249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9d793-ef61-41e2-b6fc-cdf53f7ae488 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:58.465592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 25/47] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:49:58.470806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91c7dd-916b-4b61-a18e-1bd81d7debe3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:58.471251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91c7dd-916b-4b61-a18e-1bd81d7debe3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:58.471388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91c7dd-916b-4b61-a18e-1bd81d7debe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:58.489146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-bd91c7dd-916b-4b61-a18e-1bd81d7debe3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:58.490309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bd91c7dd-916b-4b61-a18e-1bd81d7debe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:58.554325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 23/48] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:49:58.577158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9d793-ef61-41e2-b6fc-cdf53f7ae488 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:58.659389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:58.677375 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9d793-ef61-41e2-b6fc-cdf53f7ae488 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:58.709649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d97c46-51db-49ae-b71b-f4b55822efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:58.710270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d97c46-51db-49ae-b71b-f4b55822efdc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:58.710544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d97c46-51db-49ae-b71b-f4b55822efdc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:58.710812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d97c46-51db-49ae-b71b-f4b55822efdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:58.721141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d97c46-51db-49ae-b71b-f4b55822efdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-93d97c46-51db-49ae-b71b-f4b55822efdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:49:58.725813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:49:58.726794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:49:58.728194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 26/49] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:49:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:49:58.733340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91c7dd-916b-4b61-a18e-1bd81d7debe3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:58.795755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:58.817147 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e38588-bad5-4af4-b16b-826ccfbcfcf3 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:58.817432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e38588-bad5-4af4-b16b-826ccfbcfcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:58.817707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e38588-bad5-4af4-b16b-826ccfbcfcf3 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:58.819119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 27/50] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:49:58.823785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ca903-0ed1-460b-a465-32f2828f4c14 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:58.824063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ca903-0ed1-460b-a465-32f2828f4c14 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:58.824333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ca903-0ed1-460b-a465-32f2828f4c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:58.830961 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91c7dd-916b-4b61-a18e-1bd81d7debe3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:58.842824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3a7ca903-0ed1-460b-a465-32f2828f4c14 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:49:58.844027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3a7ca903-0ed1-460b-a465-32f2828f4c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:49:58.864810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c15db-e11e-4a15-883e-43eaf70f8f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:58.865417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c15db-e11e-4a15-883e-43eaf70f8f93 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:58.865692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c15db-e11e-4a15-883e-43eaf70f8f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:58.865957 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c15db-e11e-4a15-883e-43eaf70f8f93 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:58.866608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c15db-e11e-4a15-883e-43eaf70f8f93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:49:58.867185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c15db-e11e-4a15-883e-43eaf70f8f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:58.868600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c15db-e11e-4a15-883e-43eaf70f8f93 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:58.876323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 24/51] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:49:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:49:58.881738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c15db-e11e-4a15-883e-43eaf70f8f93 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:58.910291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 28/52] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:49:58.933076 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ca903-0ed1-460b-a465-32f2828f4c14 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:58.979357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c15db-e11e-4a15-883e-43eaf70f8f93 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:59.003942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:59.006794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:59.013722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7a965-695d-4c33-859b-22861b4af1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:59.014382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7a965-695d-4c33-859b-22861b4af1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:59.014662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7a965-695d-4c33-859b-22861b4af1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:59.014945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7a965-695d-4c33-859b-22861b4af1d6 None admin] PATH_INFO: `/v3/users/2ff7daf364734feb8aab110b9b2ef63b` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:59.016090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7a965-695d-4c33-859b-22861b4af1d6 None admin] RBAC: Authorizing `identity:get_user(user_id=2ff7daf364734feb8aab110b9b2ef63b)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:59.017521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7a965-695d-4c33-859b-22861b4af1d6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:59.019287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 25/53] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:49:58 2022] GET /identity/v3/users/2ff7daf364734feb8aab110b9b2ef63b => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:59.025050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7a965-695d-4c33-859b-22861b4af1d6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:59.033820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ca903-0ed1-460b-a465-32f2828f4c14 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:59.068733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cfbc5-5d7e-4753-bb15-dee5fd4e1deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:59.069368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cfbc5-5d7e-4753-bb15-dee5fd4e1deb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:59.069652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cfbc5-5d7e-4753-bb15-dee5fd4e1deb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:59.069927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cfbc5-5d7e-4753-bb15-dee5fd4e1deb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:59.072245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cfbc5-5d7e-4753-bb15-dee5fd4e1deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:59.073017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cfbc5-5d7e-4753-bb15-dee5fd4e1deb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:59.074568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 29/54] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:49:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:49:59.080058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cfbc5-5d7e-4753-bb15-dee5fd4e1deb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:59.123871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7a965-695d-4c33-859b-22861b4af1d6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:59.157757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87704fa9-1067-4241-89e3-cf40e145a43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:59.158372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87704fa9-1067-4241-89e3-cf40e145a43b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:59.158648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87704fa9-1067-4241-89e3-cf40e145a43b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:59.158917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87704fa9-1067-4241-89e3-cf40e145a43b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:59.160107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87704fa9-1067-4241-89e3-cf40e145a43b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:59.161538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87704fa9-1067-4241-89e3-cf40e145a43b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:59.163089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 26/55] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:49:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:59.168877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87704fa9-1067-4241-89e3-cf40e145a43b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:49:59.178776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cfbc5-5d7e-4753-bb15-dee5fd4e1deb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:59.212137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9905e4-09aa-4215-8028-12089dabe872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:59.212677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9905e4-09aa-4215-8028-12089dabe872 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:59.212956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9905e4-09aa-4215-8028-12089dabe872 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:59.213223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9905e4-09aa-4215-8028-12089dabe872 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:59.213859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9905e4-09aa-4215-8028-12089dabe872 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:59.215302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9905e4-09aa-4215-8028-12089dabe872 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:59.230181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:59.232637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:49:59.269303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87704fa9-1067-4241-89e3-cf40e145a43b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:49:59.302696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948feaeb-a3ca-4e7d-b0e3-9e606a03a6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:49:59.303323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948feaeb-a3ca-4e7d-b0e3-9e606a03a6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:49:59.303603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948feaeb-a3ca-4e7d-b0e3-9e606a03a6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:49:59.303873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948feaeb-a3ca-4e7d-b0e3-9e606a03a6b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:49:59.304581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948feaeb-a3ca-4e7d-b0e3-9e606a03a6b8 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.domain.id=default, user.id=2ff7daf364734feb8aab110b9b2ef63b) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:49:59.304895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948feaeb-a3ca-4e7d-b0e3-9e606a03a6b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:49:59.306846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948feaeb-a3ca-4e7d-b0e3-9e606a03a6b8 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:49:59.312522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 27/56] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Aug 10 09:49:59 2022] GET /identity/v3/role_assignments?user.id=2ff7daf364734feb8aab110b9b2ef63b&scope.domain.id=default&role.id=8417343e9ef64991bcf51bb9692da102 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:49:59.350810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 30/57] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Aug 10 09:49:59 2022] POST /identity/v3/projects => generated 298 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:50:01.370340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db1586-3111-48e2-b4b1-16bb87cda1e5 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:01.370861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db1586-3111-48e2-b4b1-16bb87cda1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:01.370946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db1586-3111-48e2-b4b1-16bb87cda1e5 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:01.372280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 28/58] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:01.377533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec147-0140-4f96-9958-c400949b2b04 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:01.377909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec147-0140-4f96-9958-c400949b2b04 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:01.378105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec147-0140-4f96-9958-c400949b2b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:01.396671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-be6ec147-0140-4f96-9958-c400949b2b04 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:01.398181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-be6ec147-0140-4f96-9958-c400949b2b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:01.462381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 31/59] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:01.486036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db1586-3111-48e2-b4b1-16bb87cda1e5 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:01.597840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db1586-3111-48e2-b4b1-16bb87cda1e5 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:01.625135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:01.630849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e720e-0fba-4d41-bcab-23e1f558483e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:01.631492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e720e-0fba-4d41-bcab-23e1f558483e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:01.631772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e720e-0fba-4d41-bcab-23e1f558483e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:01.632046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e720e-0fba-4d41-bcab-23e1f558483e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:01.633143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e720e-0fba-4d41-bcab-23e1f558483e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:01.634620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e720e-0fba-4d41-bcab-23e1f558483e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:01.636194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 29/60] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:01.641662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec147-0140-4f96-9958-c400949b2b04 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:01.735310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec147-0140-4f96-9958-c400949b2b04 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:01.757231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:01.768070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92b96f-3dbc-441c-890f-7d54af43215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:01.768664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92b96f-3dbc-441c-890f-7d54af43215a None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:01.768942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92b96f-3dbc-441c-890f-7d54af43215a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:01.769214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92b96f-3dbc-441c-890f-7d54af43215a None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:01.769853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92b96f-3dbc-441c-890f-7d54af43215a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:01.771437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92b96f-3dbc-441c-890f-7d54af43215a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:01.797327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 32/61] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:01 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:50:04.869196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de15089-a506-45dd-b6a2-2e1875bce22c None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:04.869196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de15089-a506-45dd-b6a2-2e1875bce22c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:04.869196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de15089-a506-45dd-b6a2-2e1875bce22c None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:04.869196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 30/62] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:04.875201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e6ee0-6fd8-4f44-a8e9-886d571dd755 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:04.876041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e6ee0-6fd8-4f44-a8e9-886d571dd755 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:04.876523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e6ee0-6fd8-4f44-a8e9-886d571dd755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:04.904736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-387e6ee0-6fd8-4f44-a8e9-886d571dd755 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:04.906247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-387e6ee0-6fd8-4f44-a8e9-886d571dd755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:04.938714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878eefc-f848-45b6-ad45-51ffb3c4bc14 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:04.939258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878eefc-f848-45b6-ad45-51ffb3c4bc14 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:04.939776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878eefc-f848-45b6-ad45-51ffb3c4bc14 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:04.941209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 31/63] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:50:04.952647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4170a0-fb9e-4c07-b7f4-53084f007bda None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:04.953306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4170a0-fb9e-4c07-b7f4-53084f007bda None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:04.953780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4170a0-fb9e-4c07-b7f4-53084f007bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:04.978098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0f4170a0-fb9e-4c07-b7f4-53084f007bda None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:04.979639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0f4170a0-fb9e-4c07-b7f4-53084f007bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:04.994288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 33/64] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:04.994288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582497a3-98ae-43a1-a390-7eea8d13dbd5 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:04.994288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582497a3-98ae-43a1-a390-7eea8d13dbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:04.994288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582497a3-98ae-43a1-a390-7eea8d13dbd5 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:04.994288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 34/65] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:05.000558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f2841-22e5-41b9-a3e8-8b6d6bf2ac27 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.002362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f2841-22e5-41b9-a3e8-8b6d6bf2ac27 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.002801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f2841-22e5-41b9-a3e8-8b6d6bf2ac27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.026399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9b0f2841-22e5-41b9-a3e8-8b6d6bf2ac27 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:05.027600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9b0f2841-22e5-41b9-a3e8-8b6d6bf2ac27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:05.080094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 32/66] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:05.081277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4170a0-fb9e-4c07-b7f4-53084f007bda None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.121804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 35/67] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:05.122911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f2841-22e5-41b9-a3e8-8b6d6bf2ac27 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.184041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:05.210744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4170a0-fb9e-4c07-b7f4-53084f007bda None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:05.238359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:05.245042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f2841-22e5-41b9-a3e8-8b6d6bf2ac27 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:05.247373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673833c8-3b41-42d4-ad1c-e94e414004ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:05.248281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673833c8-3b41-42d4-ad1c-e94e414004ad None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.248792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673833c8-3b41-42d4-ad1c-e94e414004ad None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.249277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673833c8-3b41-42d4-ad1c-e94e414004ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.260659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673833c8-3b41-42d4-ad1c-e94e414004ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-673833c8-3b41-42d4-ad1c-e94e414004ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:05.265530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:05.266431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:05.267229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:05.269460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 33/68] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:05.275814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673833c8-3b41-42d4-ad1c-e94e414004ad None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.283531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e76d0d-3def-4103-8e92-9acce11f4b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:05.284176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e76d0d-3def-4103-8e92-9acce11f4b68 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.284458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e76d0d-3def-4103-8e92-9acce11f4b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.284734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e76d0d-3def-4103-8e92-9acce11f4b68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.295996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e76d0d-3def-4103-8e92-9acce11f4b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-89e76d0d-3def-4103-8e92-9acce11f4b68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:05.301125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:05.301948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:05.302699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:05.307599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 36/69] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:05.308436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e76d0d-3def-4103-8e92-9acce11f4b68 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.382464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673833c8-3b41-42d4-ad1c-e94e414004ad None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:05.412782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e76d0d-3def-4103-8e92-9acce11f4b68 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:05.414792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:05.424256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f8f92-d366-4ce4-b8ea-2729c390ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:05.424884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f8f92-d366-4ce4-b8ea-2729c390ddf9 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.425158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f8f92-d366-4ce4-b8ea-2729c390ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.425426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f8f92-d366-4ce4-b8ea-2729c390ddf9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.436270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f8f92-d366-4ce4-b8ea-2729c390ddf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-f28f8f92-d366-4ce4-b8ea-2729c390ddf9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:05.441013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:05.441846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:05.442610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 34/70] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:05.443274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f8f92-d366-4ce4-b8ea-2729c390ddf9 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.456053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597d2c2-506f-4636-af29-74121b0b93cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:05.456871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597d2c2-506f-4636-af29-74121b0b93cc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.457150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597d2c2-506f-4636-af29-74121b0b93cc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.457421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597d2c2-506f-4636-af29-74121b0b93cc None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.458091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597d2c2-506f-4636-af29-74121b0b93cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:05.458397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597d2c2-506f-4636-af29-74121b0b93cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:05.460016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597d2c2-506f-4636-af29-74121b0b93cc None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:05.468491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 37/71] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:05.469675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33c4b4-241a-41f1-bf70-bf95915c44eb None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.469960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33c4b4-241a-41f1-bf70-bf95915c44eb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.470234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33c4b4-241a-41f1-bf70-bf95915c44eb None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.471675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 38/72] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:05.476476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33c4b4-241a-41f1-bf70-bf95915c44eb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.498107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:05.554190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f8f92-d366-4ce4-b8ea-2729c390ddf9 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:05.588425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33c4b4-241a-41f1-bf70-bf95915c44eb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:05.590106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc998e20-3e87-49bb-b9f1-c1d52a343c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:05.591410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc998e20-3e87-49bb-b9f1-c1d52a343c80 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.591410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc998e20-3e87-49bb-b9f1-c1d52a343c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.591410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc998e20-3e87-49bb-b9f1-c1d52a343c80 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.591962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc998e20-3e87-49bb-b9f1-c1d52a343c80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:05.592266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc998e20-3e87-49bb-b9f1-c1d52a343c80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:05.593662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc998e20-3e87-49bb-b9f1-c1d52a343c80 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:05.603143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 35/73] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:05.603752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc998e20-3e87-49bb-b9f1-c1d52a343c80 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.635548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caf7b3-13e2-4ff1-b825-c785d4d9213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:05.636593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caf7b3-13e2-4ff1-b825-c785d4d9213d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.637014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caf7b3-13e2-4ff1-b825-c785d4d9213d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.637438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caf7b3-13e2-4ff1-b825-c785d4d9213d None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.638470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caf7b3-13e2-4ff1-b825-c785d4d9213d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:05.639373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caf7b3-13e2-4ff1-b825-c785d4d9213d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:05.640719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caf7b3-13e2-4ff1-b825-c785d4d9213d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:05.653746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 39/74] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:05.653746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97fff4-882f-4589-83c3-3f668d9fec85 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.653746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97fff4-882f-4589-83c3-3f668d9fec85 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.653746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97fff4-882f-4589-83c3-3f668d9fec85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.669715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-df97fff4-882f-4589-83c3-3f668d9fec85 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:05.670900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-df97fff4-882f-4589-83c3-3f668d9fec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:05.688834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:05.710462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc998e20-3e87-49bb-b9f1-c1d52a343c80 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:05.721154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:05.738498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 40/75] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:05.739372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97fff4-882f-4589-83c3-3f668d9fec85 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.751253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9536e-21de-4271-a7f5-b57b6b7e7f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:05.751844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9536e-21de-4271-a7f5-b57b6b7e7f57 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.752128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9536e-21de-4271-a7f5-b57b6b7e7f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.752409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9536e-21de-4271-a7f5-b57b6b7e7f57 None admin] PATH_INFO: `/v3/users/2ff7daf364734feb8aab110b9b2ef63b` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.753641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9536e-21de-4271-a7f5-b57b6b7e7f57 None admin] RBAC: Authorizing `identity:get_user(user_id=2ff7daf364734feb8aab110b9b2ef63b)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:05.755163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9536e-21de-4271-a7f5-b57b6b7e7f57 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:05.761412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 36/76] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/users/2ff7daf364734feb8aab110b9b2ef63b => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:05.761412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9536e-21de-4271-a7f5-b57b6b7e7f57 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.843115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97fff4-882f-4589-83c3-3f668d9fec85 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:05.858592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9536e-21de-4271-a7f5-b57b6b7e7f57 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:05.875817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43012c8b-3ad4-4a14-a9d7-37c86ee82ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:05.876445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012c8b-3ad4-4a14-a9d7-37c86ee82ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.876724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012c8b-3ad4-4a14-a9d7-37c86ee82ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.876999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012c8b-3ad4-4a14-a9d7-37c86ee82ca0 None admin] PATH_INFO: `/v3/users/acbec83fb2f04975b8e667d4223dfe33` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.891538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267b613-283e-4357-a4f9-3f17c7a6f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:05.891960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43012c8b-3ad4-4a14-a9d7-37c86ee82ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=acbec83fb2f04975b8e667d4223dfe33)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:05.892170 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267b613-283e-4357-a4f9-3f17c7a6f398 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.892381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267b613-283e-4357-a4f9-3f17c7a6f398 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.892683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267b613-283e-4357-a4f9-3f17c7a6f398 None admin] PATH_INFO: `/v3/users/acbec83fb2f04975b8e667d4223dfe33` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.893345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43012c8b-3ad4-4a14-a9d7-37c86ee82ca0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:05.893810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267b613-283e-4357-a4f9-3f17c7a6f398 None admin] RBAC: Authorizing `identity:get_user(user_id=acbec83fb2f04975b8e667d4223dfe33)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:05.894938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 41/77] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/users/acbec83fb2f04975b8e667d4223dfe33 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:05.896086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e189a9d-3613-46fd-bd4c-0d5e4245fd14 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:05.896365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e189a9d-3613-46fd-bd4c-0d5e4245fd14 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:05.896638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e189a9d-3613-46fd-bd4c-0d5e4245fd14 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:05.897833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 42/78] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:05.898522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e189a9d-3613-46fd-bd4c-0d5e4245fd14 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.899793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267b613-283e-4357-a4f9-3f17c7a6f398 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:05.906185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 37/79] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/users/acbec83fb2f04975b8e667d4223dfe33 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:05.906185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267b613-283e-4357-a4f9-3f17c7a6f398 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:05.915929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:05.949579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.000207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e189a9d-3613-46fd-bd4c-0d5e4245fd14 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.001526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267b613-283e-4357-a4f9-3f17c7a6f398 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.034236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad7f5c-913a-4213-8663-a5dd259f5d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.034423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957981-bffc-43db-af61-d218f771bf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.034861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad7f5c-913a-4213-8663-a5dd259f5d64 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.035049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957981-bffc-43db-af61-d218f771bf80 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.035181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad7f5c-913a-4213-8663-a5dd259f5d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.035327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957981-bffc-43db-af61-d218f771bf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.035456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad7f5c-913a-4213-8663-a5dd259f5d64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.035600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957981-bffc-43db-af61-d218f771bf80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.045288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad7f5c-913a-4213-8663-a5dd259f5d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ccad7f5c-913a-4213-8663-a5dd259f5d64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:06.050021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:06.050862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:06.051652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 43/80] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:06.052301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad7f5c-913a-4213-8663-a5dd259f5d64 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.104046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957981-bffc-43db-af61-d218f771bf80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:06.105517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957981-bffc-43db-af61-d218f771bf80 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:06.121435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.150552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.150797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad7f5c-913a-4213-8663-a5dd259f5d64 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.184684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5292e-d3cc-41db-8a73-38c9a8cfb385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.185295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5292e-d3cc-41db-8a73-38c9a8cfb385 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.185582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5292e-d3cc-41db-8a73-38c9a8cfb385 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.185859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5292e-d3cc-41db-8a73-38c9a8cfb385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.207934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 38/81] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:06.209167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e1a4e-9ca5-4c8d-818c-4e8daa757137 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.209452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e1a4e-9ca5-4c8d-818c-4e8daa757137 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.209727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e1a4e-9ca5-4c8d-818c-4e8daa757137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.226308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-588e1a4e-9ca5-4c8d-818c-4e8daa757137 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:06.227561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-588e1a4e-9ca5-4c8d-818c-4e8daa757137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:06.259288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5292e-d3cc-41db-8a73-38c9a8cfb385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:06.260756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5292e-d3cc-41db-8a73-38c9a8cfb385 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:06.291473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 39/82] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:06.292373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e1a4e-9ca5-4c8d-818c-4e8daa757137 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.336023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.365447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 44/83] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:06.365999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.366526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5292e-d3cc-41db-8a73-38c9a8cfb385 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.393848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e1a4e-9ca5-4c8d-818c-4e8daa757137 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.429384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7e5c5-1315-4fcd-9c00-8387b1d3feaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.429988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7e5c5-1315-4fcd-9c00-8387b1d3feaa None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.430754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7e5c5-1315-4fcd-9c00-8387b1d3feaa None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.431055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7e5c5-1315-4fcd-9c00-8387b1d3feaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.465323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5292e-d3cc-41db-8a73-38c9a8cfb385 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.498265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7e5c5-1315-4fcd-9c00-8387b1d3feaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:06.499772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7e5c5-1315-4fcd-9c00-8387b1d3feaa None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:06.499876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615ee6c-8b82-4773-9a9c-cf641b6f240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.500372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ee6c-8b82-4773-9a9c-cf641b6f240e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.500650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ee6c-8b82-4773-9a9c-cf641b6f240e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.500920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ee6c-8b82-4773-9a9c-cf641b6f240e None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.501570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615ee6c-8b82-4773-9a9c-cf641b6f240e None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:06.501869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615ee6c-8b82-4773-9a9c-cf641b6f240e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:06.503242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615ee6c-8b82-4773-9a9c-cf641b6f240e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:06.511080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 45/84] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:06.511854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615ee6c-8b82-4773-9a9c-cf641b6f240e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.544354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.577062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.599713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 40/85] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:06.600561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7e5c5-1315-4fcd-9c00-8387b1d3feaa None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.608579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615ee6c-8b82-4773-9a9c-cf641b6f240e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.641327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da00dc-76a7-4132-ac60-04e4757b2b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.641953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da00dc-76a7-4132-ac60-04e4757b2b27 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.642230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da00dc-76a7-4132-ac60-04e4757b2b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.642505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da00dc-76a7-4132-ac60-04e4757b2b27 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.643649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da00dc-76a7-4132-ac60-04e4757b2b27 None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:06.645089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da00dc-76a7-4132-ac60-04e4757b2b27 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:06.646699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 46/86] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:06 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:06.647491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da00dc-76a7-4132-ac60-04e4757b2b27 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.697034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7e5c5-1315-4fcd-9c00-8387b1d3feaa None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.730415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ccf8b-c2f0-45b4-99af-66bd3fbb1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.731066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ccf8b-c2f0-45b4-99af-66bd3fbb1a46 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.731339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ccf8b-c2f0-45b4-99af-66bd3fbb1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.731602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ccf8b-c2f0-45b4-99af-66bd3fbb1a46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.732710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ccf8b-c2f0-45b4-99af-66bd3fbb1a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:06.734159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ccf8b-c2f0-45b4-99af-66bd3fbb1a46 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:06.735744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 41/87] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:06.736507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ccf8b-c2f0-45b4-99af-66bd3fbb1a46 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.749509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da00dc-76a7-4132-ac60-04e4757b2b27 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.751151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.783086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af71d3b-586c-449e-864a-415162ab0286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.783714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af71d3b-586c-449e-864a-415162ab0286 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.783991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af71d3b-586c-449e-864a-415162ab0286 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.784265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af71d3b-586c-449e-864a-415162ab0286 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.785372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af71d3b-586c-449e-864a-415162ab0286 None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:06.785909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.786862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af71d3b-586c-449e-864a-415162ab0286 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:06.788649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 47/88] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:06 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:06.789243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af71d3b-586c-449e-864a-415162ab0286 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.841377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ccf8b-c2f0-45b4-99af-66bd3fbb1a46 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.874145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410c6a4-3571-4072-a0ff-852a9b7e5e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.874770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410c6a4-3571-4072-a0ff-852a9b7e5e0e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.875078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410c6a4-3571-4072-a0ff-852a9b7e5e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.875359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410c6a4-3571-4072-a0ff-852a9b7e5e0e None admin] PATH_INFO: `/v3/users/acbec83fb2f04975b8e667d4223dfe33` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.876487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410c6a4-3571-4072-a0ff-852a9b7e5e0e None admin] RBAC: Authorizing `identity:get_user(user_id=acbec83fb2f04975b8e667d4223dfe33)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:06.877922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410c6a4-3571-4072-a0ff-852a9b7e5e0e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:06.879521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 42/89] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:06 2022] GET /identity/v3/users/acbec83fb2f04975b8e667d4223dfe33 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:06.880283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410c6a4-3571-4072-a0ff-852a9b7e5e0e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.884720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af71d3b-586c-449e-864a-415162ab0286 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.917325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952e272-631e-47e0-b5bf-d5e19703d43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:06.917937 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952e272-631e-47e0-b5bf-d5e19703d43e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:06.918214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952e272-631e-47e0-b5bf-d5e19703d43e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:06.918481 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952e272-631e-47e0-b5bf-d5e19703d43e None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:06.919598 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952e272-631e-47e0-b5bf-d5e19703d43e None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:06.921015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952e272-631e-47e0-b5bf-d5e19703d43e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:06.922609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 48/90] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:06 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:06.923365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952e272-631e-47e0-b5bf-d5e19703d43e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:06.960921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:06.981485 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410c6a4-3571-4072-a0ff-852a9b7e5e0e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:06.994293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:07.014489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3557a18-6d19-448e-b7b5-b737dde2bf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:07.015117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3557a18-6d19-448e-b7b5-b737dde2bf44 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:07.015399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3557a18-6d19-448e-b7b5-b737dde2bf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:07.015668 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3557a18-6d19-448e-b7b5-b737dde2bf44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:07.016386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3557a18-6d19-448e-b7b5-b737dde2bf44 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5, user.id=2ff7daf364734feb8aab110b9b2ef63b) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:07.016689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3557a18-6d19-448e-b7b5-b737dde2bf44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:07.018123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3557a18-6d19-448e-b7b5-b737dde2bf44 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:07.019533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952e272-631e-47e0-b5bf-d5e19703d43e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:07.023322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 43/91] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:06 2022] GET /identity/v3/role_assignments?user.id=2ff7daf364734feb8aab110b9b2ef63b&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=8417343e9ef64991bcf51bb9692da102 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:07.024673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3557a18-6d19-448e-b7b5-b737dde2bf44 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:07.057495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1eeefc-7cbd-4246-9de6-587415801edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:07.058181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1eeefc-7cbd-4246-9de6-587415801edb None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:07.058463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1eeefc-7cbd-4246-9de6-587415801edb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:07.058735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1eeefc-7cbd-4246-9de6-587415801edb None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:07.059502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1eeefc-7cbd-4246-9de6-587415801edb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:07.060995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1eeefc-7cbd-4246-9de6-587415801edb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:07.088212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 49/92] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:06 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:07.089909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1eeefc-7cbd-4246-9de6-587415801edb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:07.166168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3557a18-6d19-448e-b7b5-b737dde2bf44 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:07.218712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591efa68-c00b-4d14-870a-d8d1f21c9261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:07.219307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591efa68-c00b-4d14-870a-d8d1f21c9261 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:07.219589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591efa68-c00b-4d14-870a-d8d1f21c9261 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:07.219865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591efa68-c00b-4d14-870a-d8d1f21c9261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:07.220615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591efa68-c00b-4d14-870a-d8d1f21c9261 None admin] RBAC: Adding query filter params (role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5, user.id=acbec83fb2f04975b8e667d4223dfe33) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:07.220929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591efa68-c00b-4d14-870a-d8d1f21c9261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:07.222534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591efa68-c00b-4d14-870a-d8d1f21c9261 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:07.228802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:07.230202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 44/93] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:07 2022] GET /identity/v3/role_assignments?user.id=acbec83fb2f04975b8e667d4223dfe33&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:07.231152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591efa68-c00b-4d14-870a-d8d1f21c9261 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:07.238173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1eeefc-7cbd-4246-9de6-587415801edb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:07.285014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977bfe8-7bb6-460e-a9db-c065e8cc7f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:07.285660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977bfe8-7bb6-460e-a9db-c065e8cc7f01 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:07.285941 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977bfe8-7bb6-460e-a9db-c065e8cc7f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:07.286212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977bfe8-7bb6-460e-a9db-c065e8cc7f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:07.292689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:07.376759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591efa68-c00b-4d14-870a-d8d1f21c9261 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:07.399678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977bfe8-7bb6-460e-a9db-c065e8cc7f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:07.404268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977bfe8-7bb6-460e-a9db-c065e8cc7f01 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:07.439448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2918-ced0-4910-ba0e-e5f8b5d48f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:07.440072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2918-ced0-4910-ba0e-e5f8b5d48f65 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:07.440358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2918-ced0-4910-ba0e-e5f8b5d48f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:07.440637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2918-ced0-4910-ba0e-e5f8b5d48f65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:07.441375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2918-ced0-4910-ba0e-e5f8b5d48f65 None admin] RBAC: Adding query filter params (role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5, user.id=acbec83fb2f04975b8e667d4223dfe33) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:07.441684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2918-ced0-4910-ba0e-e5f8b5d48f65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:07.443177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2918-ced0-4910-ba0e-e5f8b5d48f65 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:07.451308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 45/94] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:07 2022] GET /identity/v3/role_assignments?user.id=acbec83fb2f04975b8e667d4223dfe33&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:07.539643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:07.549304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 50/95] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:07.555680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2918-ced0-4910-ba0e-e5f8b5d48f65 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:07.684858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2918-ced0-4910-ba0e-e5f8b5d48f65 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:07.687257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:07.726393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe906ac3-14f5-4b28-8448-1362ffe3a758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:07.727132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe906ac3-14f5-4b28-8448-1362ffe3a758 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:07.727413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe906ac3-14f5-4b28-8448-1362ffe3a758 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:07.727684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe906ac3-14f5-4b28-8448-1362ffe3a758 None admin] PATH_INFO: `/v3/projects/60292cebe3104fc683cb7b1c8cff3f07` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:07.729515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe906ac3-14f5-4b28-8448-1362ffe3a758 None admin] RBAC: Authorizing `identity:get_project(project_id=60292cebe3104fc683cb7b1c8cff3f07)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:07.731000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe906ac3-14f5-4b28-8448-1362ffe3a758 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:07.732827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 46/96] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:07 2022] GET /identity/v3/projects/60292cebe3104fc683cb7b1c8cff3f07 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:07.738561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977bfe8-7bb6-460e-a9db-c065e8cc7f01 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:07.856912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977bfe8-7bb6-460e-a9db-c065e8cc7f01 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:07.903681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de68f33-9e5a-4674-9f99-887f96b152aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:07.905099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de68f33-9e5a-4674-9f99-887f96b152aa None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:07.905597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de68f33-9e5a-4674-9f99-887f96b152aa None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:07.906069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de68f33-9e5a-4674-9f99-887f96b152aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:07.907058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de68f33-9e5a-4674-9f99-887f96b152aa None admin] RBAC: Adding query filter params (role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=60292cebe3104fc683cb7b1c8cff3f07, user.id=acbec83fb2f04975b8e667d4223dfe33) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:07.907572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de68f33-9e5a-4674-9f99-887f96b152aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:07.909229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de68f33-9e5a-4674-9f99-887f96b152aa None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:07.918276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 51/97] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:07 2022] GET /identity/v3/role_assignments?user.id=acbec83fb2f04975b8e667d4223dfe33&scope.project.id=60292cebe3104fc683cb7b1c8cff3f07&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:09.706248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eddd6-1de0-4965-a8c2-8aead147e649 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:09.706725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eddd6-1de0-4965-a8c2-8aead147e649 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:09.706725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eddd6-1de0-4965-a8c2-8aead147e649 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:09.708310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 47/98] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:09.713639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaf8f5-8444-4ed9-9c9f-eeae33b428be None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:09.714028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaf8f5-8444-4ed9-9c9f-eeae33b428be None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:09.714204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaf8f5-8444-4ed9-9c9f-eeae33b428be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:09.732776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b6aaf8f5-8444-4ed9-9c9f-eeae33b428be None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:09.733970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b6aaf8f5-8444-4ed9-9c9f-eeae33b428be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:09.788105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:09.807339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 52/99] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:09.824589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1eddd6-1de0-4965-a8c2-8aead147e649 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:09.925491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a67731-3ec3-464c-bd02-1a9c0ac719d0 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:09.925831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a67731-3ec3-464c-bd02-1a9c0ac719d0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:09.926039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a67731-3ec3-464c-bd02-1a9c0ac719d0 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:09.928078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 53/100] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:50:09.933334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca52285-d00d-4936-88f2-5e7f6a82d8cd None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:09.933629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca52285-d00d-4936-88f2-5e7f6a82d8cd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:09.933908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca52285-d00d-4936-88f2-5e7f6a82d8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:09.935072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1eddd6-1de0-4965-a8c2-8aead147e649 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:09.953938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dca52285-d00d-4936-88f2-5e7f6a82d8cd None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:09.955646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dca52285-d00d-4936-88f2-5e7f6a82d8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:09.987261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248289c-2b23-479e-92ec-9edadf951055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:09.988291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248289c-2b23-479e-92ec-9edadf951055 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:09.988759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248289c-2b23-479e-92ec-9edadf951055 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:09.989219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248289c-2b23-479e-92ec-9edadf951055 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.019407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248289c-2b23-479e-92ec-9edadf951055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-0248289c-2b23-479e-92ec-9edadf951055 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:10.028704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:10.029532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:10.030268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 48/101] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:50:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:10.035864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248289c-2b23-479e-92ec-9edadf951055 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:10.067280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 54/102] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:10.072011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:10.100518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca52285-d00d-4936-88f2-5e7f6a82d8cd None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:10.143797 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248289c-2b23-479e-92ec-9edadf951055 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:10.191876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00660494-3385-4c49-b25d-b828d343478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:10.192909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00660494-3385-4c49-b25d-b828d343478c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.193368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00660494-3385-4c49-b25d-b828d343478c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.193906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00660494-3385-4c49-b25d-b828d343478c None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.196213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00660494-3385-4c49-b25d-b828d343478c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:10.196526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00660494-3385-4c49-b25d-b828d343478c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:10.197915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00660494-3385-4c49-b25d-b828d343478c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:10.215723 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 49/103] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:10.220731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00660494-3385-4c49-b25d-b828d343478c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:10.301717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca52285-d00d-4936-88f2-5e7f6a82d8cd None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:10.340767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:10.359035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d44b03-83bb-47e3-a5cb-7297c57ec4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:10.359778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d44b03-83bb-47e3-a5cb-7297c57ec4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.360126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d44b03-83bb-47e3-a5cb-7297c57ec4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.360467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d44b03-83bb-47e3-a5cb-7297c57ec4e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.372513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d44b03-83bb-47e3-a5cb-7297c57ec4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-d6d44b03-83bb-47e3-a5cb-7297c57ec4e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:10.378309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:10.379334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:10.380084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:10.380851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 55/104] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:10.387465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14afe51-fff0-403b-8024-7ef890f32999 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.387773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14afe51-fff0-403b-8024-7ef890f32999 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.388055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14afe51-fff0-403b-8024-7ef890f32999 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.389971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 56/105] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:10.394228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14afe51-fff0-403b-8024-7ef890f32999 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:10.423338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:10.435667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00660494-3385-4c49-b25d-b828d343478c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:10.501531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4e8d-9ad1-4bd1-98a8-f1fe36dbd82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:10.503592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4e8d-9ad1-4bd1-98a8-f1fe36dbd82d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.503878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4e8d-9ad1-4bd1-98a8-f1fe36dbd82d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.504154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4e8d-9ad1-4bd1-98a8-f1fe36dbd82d None admin] PATH_INFO: `/v3/users/2ff7daf364734feb8aab110b9b2ef63b` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.505473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b4e8d-9ad1-4bd1-98a8-f1fe36dbd82d None admin] RBAC: Authorizing `identity:get_user(user_id=2ff7daf364734feb8aab110b9b2ef63b)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:10.506931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b4e8d-9ad1-4bd1-98a8-f1fe36dbd82d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:10.510437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 50/106] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:10 2022] GET /identity/v3/users/2ff7daf364734feb8aab110b9b2ef63b => generated 247 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:10.514009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828569f-49d1-4c08-a2ce-c05834e4fa78 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.515239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828569f-49d1-4c08-a2ce-c05834e4fa78 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.515519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828569f-49d1-4c08-a2ce-c05834e4fa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.543625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14afe51-fff0-403b-8024-7ef890f32999 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:10.556599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-8828569f-49d1-4c08-a2ce-c05834e4fa78 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:10.557998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8828569f-49d1-4c08-a2ce-c05834e4fa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:10.607194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3710ad4-76a0-4206-9b8b-4a8904c5a4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:10.608130 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710ad4-76a0-4206-9b8b-4a8904c5a4cd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.608407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710ad4-76a0-4206-9b8b-4a8904c5a4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.608679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710ad4-76a0-4206-9b8b-4a8904c5a4cd None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.609347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3710ad4-76a0-4206-9b8b-4a8904c5a4cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:10.609650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3710ad4-76a0-4206-9b8b-4a8904c5a4cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:10.611738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3710ad4-76a0-4206-9b8b-4a8904c5a4cd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:10.622645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 57/107] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:10.623584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3710ad4-76a0-4206-9b8b-4a8904c5a4cd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:10.654272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 51/108] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:10.655743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f04c0-dc1f-4218-8b48-8efdfe673c35 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.656174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f04c0-dc1f-4218-8b48-8efdfe673c35 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.656571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f04c0-dc1f-4218-8b48-8efdfe673c35 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.657938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 52/109] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:10.658781 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f04c0-dc1f-4218-8b48-8efdfe673c35 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:10.691912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:10.767641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3710ad4-76a0-4206-9b8b-4a8904c5a4cd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:10.792761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:10.807117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f04c0-dc1f-4218-8b48-8efdfe673c35 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:10.812674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c2cb1-ddb9-46dd-b1e2-1241185ca92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:10.813513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c2cb1-ddb9-46dd-b1e2-1241185ca92d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.813949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c2cb1-ddb9-46dd-b1e2-1241185ca92d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.814367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c2cb1-ddb9-46dd-b1e2-1241185ca92d None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.815708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c2cb1-ddb9-46dd-b1e2-1241185ca92d None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:10.817813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c2cb1-ddb9-46dd-b1e2-1241185ca92d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:10.820087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 58/110] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:10 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:10.822116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e63cc1-18ef-4399-8aaf-4a7df5bc4525 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.822564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e63cc1-18ef-4399-8aaf-4a7df5bc4525 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.823021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e63cc1-18ef-4399-8aaf-4a7df5bc4525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.845946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b1e63cc1-18ef-4399-8aaf-4a7df5bc4525 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:10.847656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b1e63cc1-18ef-4399-8aaf-4a7df5bc4525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:10.853999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa3735-8569-4a5a-ab9e-4a0a0e07e0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:10.854754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa3735-8569-4a5a-ab9e-4a0a0e07e0fe None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:10.855122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa3735-8569-4a5a-ab9e-4a0a0e07e0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:10.855553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa3735-8569-4a5a-ab9e-4a0a0e07e0fe None admin] PATH_INFO: `/v3/users/acbec83fb2f04975b8e667d4223dfe33` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:10.856828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa3735-8569-4a5a-ab9e-4a0a0e07e0fe None admin] RBAC: Authorizing `identity:get_user(user_id=acbec83fb2f04975b8e667d4223dfe33)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:10.858985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa3735-8569-4a5a-ab9e-4a0a0e07e0fe None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:10.863098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 53/111] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:10 2022] GET /identity/v3/users/acbec83fb2f04975b8e667d4223dfe33 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:10.864098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa3735-8569-4a5a-ab9e-4a0a0e07e0fe None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:10.927155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 59/112] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:10.939155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e63cc1-18ef-4399-8aaf-4a7df5bc4525 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:10.967987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:10.968886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa3735-8569-4a5a-ab9e-4a0a0e07e0fe None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.002810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92817ef5-2ba0-4102-a937-c8bb6083d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.003503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92817ef5-2ba0-4102-a937-c8bb6083d856 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.003788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92817ef5-2ba0-4102-a937-c8bb6083d856 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.004068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92817ef5-2ba0-4102-a937-c8bb6083d856 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.013941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92817ef5-2ba0-4102-a937-c8bb6083d856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-92817ef5-2ba0-4102-a937-c8bb6083d856 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:11.018511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:11.019581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 54/113] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:11.020546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92817ef5-2ba0-4102-a937-c8bb6083d856 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.028776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e63cc1-18ef-4399-8aaf-4a7df5bc4525 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.041890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:11.062557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49725b83-0527-44e5-9747-7e2c21b40126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.063175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49725b83-0527-44e5-9747-7e2c21b40126 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.063453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49725b83-0527-44e5-9747-7e2c21b40126 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.063725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49725b83-0527-44e5-9747-7e2c21b40126 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/users/2ff7daf364734feb8aab110b9b2ef63b/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.065578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49725b83-0527-44e5-9747-7e2c21b40126 None admin] RBAC: Authorizing `identity:create_grant(project_id=02ab39fdbee942d8bd432a27ca1d13c5, user_id=2ff7daf364734feb8aab110b9b2ef63b, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:11.067052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49725b83-0527-44e5-9747-7e2c21b40126 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:11.081454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 60/114] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:50:10 2022] PUT /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/users/2ff7daf364734feb8aab110b9b2ef63b/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:11.082608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cfd5e-253f-4388-9136-a55734745a72 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.082887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cfd5e-253f-4388-9136-a55734745a72 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.083188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cfd5e-253f-4388-9136-a55734745a72 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.084433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 61/115] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:11.085482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f9e61-6a57-412c-b342-2dfbca999f76 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.085762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f9e61-6a57-412c-b342-2dfbca999f76 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.086049 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f9e61-6a57-412c-b342-2dfbca999f76 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.087577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 62/116] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:11.092118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f9e61-6a57-412c-b342-2dfbca999f76 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.134603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92817ef5-2ba0-4102-a937-c8bb6083d856 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.174221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba42e3-afb6-48b0-8073-c21573a9a0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.174892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba42e3-afb6-48b0-8073-c21573a9a0cb None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.175194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba42e3-afb6-48b0-8073-c21573a9a0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.175467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba42e3-afb6-48b0-8073-c21573a9a0cb None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.176712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba42e3-afb6-48b0-8073-c21573a9a0cb None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:11.178207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba42e3-afb6-48b0-8073-c21573a9a0cb None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:11.179939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 55/117] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:11 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:11.181767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba42e3-afb6-48b0-8073-c21573a9a0cb None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.211548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:11.218584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f9e61-6a57-412c-b342-2dfbca999f76 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.264667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7a1f4-01d1-43eb-97e2-0f9560358140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.266199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7a1f4-01d1-43eb-97e2-0f9560358140 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.266509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7a1f4-01d1-43eb-97e2-0f9560358140 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.266836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7a1f4-01d1-43eb-97e2-0f9560358140 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.279625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7a1f4-01d1-43eb-97e2-0f9560358140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:11.282804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-41c7a1f4-01d1-43eb-97e2-0f9560358140 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:11.285288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:11.286421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:11.287595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 63/118] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:11.288419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534ab1f-c599-4bf8-963b-6adaefd5de82 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.288663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534ab1f-c599-4bf8-963b-6adaefd5de82 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.289889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534ab1f-c599-4bf8-963b-6adaefd5de82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.299905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba42e3-afb6-48b0-8073-c21573a9a0cb None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.310802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5534ab1f-c599-4bf8-963b-6adaefd5de82 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:11.312324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5534ab1f-c599-4bf8-963b-6adaefd5de82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:11.339010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b1343-6ec5-4007-8874-cb5e5981bf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.339847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b1343-6ec5-4007-8874-cb5e5981bf3a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.340374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b1343-6ec5-4007-8874-cb5e5981bf3a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.340656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b1343-6ec5-4007-8874-cb5e5981bf3a None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.341464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b1343-6ec5-4007-8874-cb5e5981bf3a None admin] RBAC: Adding query filter params (name=member) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:11.341773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b1343-6ec5-4007-8874-cb5e5981bf3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:11.343746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b1343-6ec5-4007-8874-cb5e5981bf3a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:11.352600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 56/119] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:11.353895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655c99e-4f0d-42e3-978c-2f53b239e776 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.354188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655c99e-4f0d-42e3-978c-2f53b239e776 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.354457 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655c99e-4f0d-42e3-978c-2f53b239e776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.371989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-9655c99e-4f0d-42e3-978c-2f53b239e776 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:11.373220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9655c99e-4f0d-42e3-978c-2f53b239e776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:11.384214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 64/120] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:11.385259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534ab1f-c599-4bf8-963b-6adaefd5de82 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.461825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 57/121] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:11.462657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655c99e-4f0d-42e3-978c-2f53b239e776 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.492774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:11.506891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534ab1f-c599-4bf8-963b-6adaefd5de82 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.546107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77caeb-160c-4764-aec2-4f75adb61b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.546772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77caeb-160c-4764-aec2-4f75adb61b66 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.547092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77caeb-160c-4764-aec2-4f75adb61b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.547364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77caeb-160c-4764-aec2-4f75adb61b66 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/users/acbec83fb2f04975b8e667d4223dfe33/roles/02c7974bceec49ee8895dabf30cc8fff` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.550211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77caeb-160c-4764-aec2-4f75adb61b66 None admin] RBAC: Authorizing `identity:create_grant(project_id=02ab39fdbee942d8bd432a27ca1d13c5, user_id=acbec83fb2f04975b8e667d4223dfe33, role_id=02c7974bceec49ee8895dabf30cc8fff)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:11.552171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77caeb-160c-4764-aec2-4f75adb61b66 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:11.553747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:11.577161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 65/122] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:50:11 2022] PUT /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/users/acbec83fb2f04975b8e667d4223dfe33/roles/02c7974bceec49ee8895dabf30cc8fff => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:11.577161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77caeb-160c-4764-aec2-4f75adb61b66 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.577434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655c99e-4f0d-42e3-978c-2f53b239e776 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.621376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698373a3-55ed-4643-852a-8021353aa73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.622705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698373a3-55ed-4643-852a-8021353aa73d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.623013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698373a3-55ed-4643-852a-8021353aa73d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.623292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698373a3-55ed-4643-852a-8021353aa73d None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.624056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698373a3-55ed-4643-852a-8021353aa73d None admin] RBAC: Adding query filter params (name=member) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:11.624380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698373a3-55ed-4643-852a-8021353aa73d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:11.625873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698373a3-55ed-4643-852a-8021353aa73d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:11.635811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 58/123] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:11.636722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698373a3-55ed-4643-852a-8021353aa73d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.729030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77caeb-160c-4764-aec2-4f75adb61b66 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.759048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698373a3-55ed-4643-852a-8021353aa73d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.773362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311ab1f-8108-496f-a0c9-96649739d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.774828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311ab1f-8108-496f-a0c9-96649739d059 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.775135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311ab1f-8108-496f-a0c9-96649739d059 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.775405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311ab1f-8108-496f-a0c9-96649739d059 None admin] PATH_INFO: `/v3/users/acbec83fb2f04975b8e667d4223dfe33` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.776994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311ab1f-8108-496f-a0c9-96649739d059 None admin] RBAC: Authorizing `identity:get_user(user_id=acbec83fb2f04975b8e667d4223dfe33)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:11.778607 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311ab1f-8108-496f-a0c9-96649739d059 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:11.781195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 66/124] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:11 2022] GET /identity/v3/users/acbec83fb2f04975b8e667d4223dfe33 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:11.781859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311ab1f-8108-496f-a0c9-96649739d059 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.799893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b271-9ee8-460f-8142-4fee7eb83f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.800490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b271-9ee8-460f-8142-4fee7eb83f21 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.800899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b271-9ee8-460f-8142-4fee7eb83f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.801321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b271-9ee8-460f-8142-4fee7eb83f21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.802404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98b271-9ee8-460f-8142-4fee7eb83f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:11.803964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98b271-9ee8-460f-8142-4fee7eb83f21 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:11.805795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 59/125] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:11.806471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b271-9ee8-460f-8142-4fee7eb83f21 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.811805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:11.819077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:11.921205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311ab1f-8108-496f-a0c9-96649739d059 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.924556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b271-9ee8-460f-8142-4fee7eb83f21 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:11.959476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2d5f5-d8a5-41b3-b26e-618b5dc6e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.960273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2d5f5-d8a5-41b3-b26e-618b5dc6e359 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.960869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2d5f5-d8a5-41b3-b26e-618b5dc6e359 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.961157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2d5f5-d8a5-41b3-b26e-618b5dc6e359 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.970469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe7f80-f70f-478d-9cfc-621f3a91a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:11.971628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7f80-f70f-478d-9cfc-621f3a91a12f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:11.972007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7f80-f70f-478d-9cfc-621f3a91a12f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:11.972375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7f80-f70f-478d-9cfc-621f3a91a12f None admin] PATH_INFO: `/v3/users/acbec83fb2f04975b8e667d4223dfe33` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:11.973772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2d5f5-d8a5-41b3-b26e-618b5dc6e359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:11.973931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe7f80-f70f-478d-9cfc-621f3a91a12f None admin] RBAC: Authorizing `identity:get_user(user_id=acbec83fb2f04975b8e667d4223dfe33)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:11.975509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe7f80-f70f-478d-9cfc-621f3a91a12f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:11.977470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 60/126] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:11 2022] GET /identity/v3/users/acbec83fb2f04975b8e667d4223dfe33 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:11.978155 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe7f80-f70f-478d-9cfc-621f3a91a12f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-15d2d5f5-d8a5-41b3-b26e-618b5dc6e359 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:11.983781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:11.984801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:11.985732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:11.986672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 67/127] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:11.986672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2d5f5-d8a5-41b3-b26e-618b5dc6e359 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:12.075920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:12.085615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:12.104973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2d5f5-d8a5-41b3-b26e-618b5dc6e359 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:12.112038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe7f80-f70f-478d-9cfc-621f3a91a12f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:12.143085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d217a3-4a20-466a-b56b-9edd635e5d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:12.143929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d217a3-4a20-466a-b56b-9edd635e5d1e None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:12.144566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d217a3-4a20-466a-b56b-9edd635e5d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:12.144974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d217a3-4a20-466a-b56b-9edd635e5d1e None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:12.145849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d217a3-4a20-466a-b56b-9edd635e5d1e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:12.147525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d217a3-4a20-466a-b56b-9edd635e5d1e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:12.153421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb3b00-b880-4bd6-8b20-83a677f82bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:12.154496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb3b00-b880-4bd6-8b20-83a677f82bed None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:12.154871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb3b00-b880-4bd6-8b20-83a677f82bed None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:12.155267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb3b00-b880-4bd6-8b20-83a677f82bed None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:12.156488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb3b00-b880-4bd6-8b20-83a677f82bed None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:12.157919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb3b00-b880-4bd6-8b20-83a677f82bed None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:12.160001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 61/128] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:11 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:12.160573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb3b00-b880-4bd6-8b20-83a677f82bed None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:12.189655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 68/129] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:50:11 2022] POST /identity/v3/projects => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:12.191552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d217a3-4a20-466a-b56b-9edd635e5d1e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:12.314382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb3b00-b880-4bd6-8b20-83a677f82bed None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:12.361245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d217a3-4a20-466a-b56b-9edd635e5d1e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:12.385987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79468e3c-569d-4cdc-bee8-3baaa648209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:12.386729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79468e3c-569d-4cdc-bee8-3baaa648209a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:12.387036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79468e3c-569d-4cdc-bee8-3baaa648209a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:12.387412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79468e3c-569d-4cdc-bee8-3baaa648209a None admin] PATH_INFO: `/v3/projects/60292cebe3104fc683cb7b1c8cff3f07` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:12.388968 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79468e3c-569d-4cdc-bee8-3baaa648209a None admin] RBAC: Authorizing `identity:get_project(project_id=60292cebe3104fc683cb7b1c8cff3f07)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:12.390732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79468e3c-569d-4cdc-bee8-3baaa648209a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:12.393206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 62/130] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:12 2022] GET /identity/v3/projects/60292cebe3104fc683cb7b1c8cff3f07 => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:12.394016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79468e3c-569d-4cdc-bee8-3baaa648209a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:12.411742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:12.420315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:12.422000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3bbb4-8573-446b-95a7-eb9b72aa3c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:12.422903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3bbb4-8573-446b-95a7-eb9b72aa3c06 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:12.423224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3bbb4-8573-446b-95a7-eb9b72aa3c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:12.423507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3bbb4-8573-446b-95a7-eb9b72aa3c06 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:12.425139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3bbb4-8573-446b-95a7-eb9b72aa3c06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:12.425758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3bbb4-8573-446b-95a7-eb9b72aa3c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:12.427786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3bbb4-8573-446b-95a7-eb9b72aa3c06 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:12.437982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 69/131] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:12.439192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3bbb4-8573-446b-95a7-eb9b72aa3c06 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:12.519145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79468e3c-569d-4cdc-bee8-3baaa648209a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:12.554320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3bbb4-8573-446b-95a7-eb9b72aa3c06 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:12.555069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f1f5-7515-4ddd-9ac6-9e1a2d9ee3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:12.555742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f1f5-7515-4ddd-9ac6-9e1a2d9ee3e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:12.556015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f1f5-7515-4ddd-9ac6-9e1a2d9ee3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:12.556286 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f1f5-7515-4ddd-9ac6-9e1a2d9ee3e0 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/users/acbec83fb2f04975b8e667d4223dfe33/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:12.559066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36f1f5-7515-4ddd-9ac6-9e1a2d9ee3e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=02ab39fdbee942d8bd432a27ca1d13c5, user_id=acbec83fb2f04975b8e667d4223dfe33, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:12.560540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36f1f5-7515-4ddd-9ac6-9e1a2d9ee3e0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:12.578237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 63/132] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:50:12 2022] PUT /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/users/acbec83fb2f04975b8e667d4223dfe33/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:12.579135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f1f5-7515-4ddd-9ac6-9e1a2d9ee3e0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:12.594556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668d392-ecb0-48c7-8e8e-1ac31d38950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:12.595371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668d392-ecb0-48c7-8e8e-1ac31d38950d None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:12.595654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668d392-ecb0-48c7-8e8e-1ac31d38950d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:12.595977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668d392-ecb0-48c7-8e8e-1ac31d38950d None admin] PATH_INFO: `/v3/projects/60292cebe3104fc683cb7b1c8cff3f07/users/acbec83fb2f04975b8e667d4223dfe33/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:12.602055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668d392-ecb0-48c7-8e8e-1ac31d38950d None admin] RBAC: Authorizing `identity:create_grant(project_id=60292cebe3104fc683cb7b1c8cff3f07, user_id=acbec83fb2f04975b8e667d4223dfe33, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:12.604098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668d392-ecb0-48c7-8e8e-1ac31d38950d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:12.632692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 70/133] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:50:12 2022] PUT /identity/v3/projects/60292cebe3104fc683cb7b1c8cff3f07/users/acbec83fb2f04975b8e667d4223dfe33/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:12.702629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:12.733402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f1f5-7515-4ddd-9ac6-9e1a2d9ee3e0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:12.772441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04772c61-7e51-474e-8578-6d7630f0a968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:12.773007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04772c61-7e51-474e-8578-6d7630f0a968 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:12.773329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04772c61-7e51-474e-8578-6d7630f0a968 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:12.773604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04772c61-7e51-474e-8578-6d7630f0a968 None admin] PATH_INFO: `/v3/users/6dcc22ec2d8e4a8d9dfc14e251b0c891` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:12.790396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04772c61-7e51-474e-8578-6d7630f0a968 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcc22ec2d8e4a8d9dfc14e251b0c891)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:12.791924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04772c61-7e51-474e-8578-6d7630f0a968 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:12.793776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 64/134] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:12 2022] GET /identity/v3/users/6dcc22ec2d8e4a8d9dfc14e251b0c891 => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:12.799454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668d392-ecb0-48c7-8e8e-1ac31d38950d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:12.863356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:12.911255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668d392-ecb0-48c7-8e8e-1ac31d38950d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:12.947878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b706015-8819-46fb-a156-ddf10a53ecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:12.948512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b706015-8819-46fb-a156-ddf10a53ecfa None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:12.948791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b706015-8819-46fb-a156-ddf10a53ecfa None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:12.949109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b706015-8819-46fb-a156-ddf10a53ecfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:13.021904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b706015-8819-46fb-a156-ddf10a53ecfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:13.023476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b706015-8819-46fb-a156-ddf10a53ecfa None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:13.078188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:13.126947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 71/135] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:13.132866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04772c61-7e51-474e-8578-6d7630f0a968 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:13.251764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04772c61-7e51-474e-8578-6d7630f0a968 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:13.287379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:13.293809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22681d46-7351-4fdb-a023-15255b73bd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:13.294645 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22681d46-7351-4fdb-a023-15255b73bd68 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:13.295142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22681d46-7351-4fdb-a023-15255b73bd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:13.295418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22681d46-7351-4fdb-a023-15255b73bd68 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:13.296555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22681d46-7351-4fdb-a023-15255b73bd68 None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:13.298000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22681d46-7351-4fdb-a023-15255b73bd68 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:13.299665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 65/136] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:13 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:13.306030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b706015-8819-46fb-a156-ddf10a53ecfa None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:13.352388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d291e-e14d-4b24-a4e6-b2646fa5d2ef None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:13.352733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d291e-e14d-4b24-a4e6-b2646fa5d2ef None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:13.353002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d291e-e14d-4b24-a4e6-b2646fa5d2ef None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:13.354775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 66/137] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:13.360082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94be1ed-cae7-41e7-bcf6-37a6e91eeb23 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:13.360366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94be1ed-cae7-41e7-bcf6-37a6e91eeb23 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:13.360641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94be1ed-cae7-41e7-bcf6-37a6e91eeb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:13.388962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-b94be1ed-cae7-41e7-bcf6-37a6e91eeb23 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:13.390139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b94be1ed-cae7-41e7-bcf6-37a6e91eeb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:13.488946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b706015-8819-46fb-a156-ddf10a53ecfa None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:13.526049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 67/138] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:13.560917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baec7ea-4afa-4e48-a240-027f9dce5735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:13.561582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baec7ea-4afa-4e48-a240-027f9dce5735 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:13.561926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baec7ea-4afa-4e48-a240-027f9dce5735 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:13.562306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baec7ea-4afa-4e48-a240-027f9dce5735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:13.563348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baec7ea-4afa-4e48-a240-027f9dce5735 None admin] RBAC: Adding query filter params (role.id=053f81c391444db1b6409c4eb953dc35, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5, user.id=6dcc22ec2d8e4a8d9dfc14e251b0c891) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:13.563793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baec7ea-4afa-4e48-a240-027f9dce5735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:13.565907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baec7ea-4afa-4e48-a240-027f9dce5735 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:13.581030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 72/139] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:13 2022] GET /identity/v3/role_assignments?user.id=6dcc22ec2d8e4a8d9dfc14e251b0c891&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=053f81c391444db1b6409c4eb953dc35 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:13.600552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94be1ed-cae7-41e7-bcf6-37a6e91eeb23 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:13.716124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94be1ed-cae7-41e7-bcf6-37a6e91eeb23 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:13.748236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:13.754980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876696e-400c-4788-9368-2c4f7bad06b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:13.755627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876696e-400c-4788-9368-2c4f7bad06b4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:13.755905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876696e-400c-4788-9368-2c4f7bad06b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:13.756175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876696e-400c-4788-9368-2c4f7bad06b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:13.767639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876696e-400c-4788-9368-2c4f7bad06b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-e876696e-400c-4788-9368-2c4f7bad06b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:13.772306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:13.773176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:13.774489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 68/140] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:50:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:13.780262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baec7ea-4afa-4e48-a240-027f9dce5735 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:13.797898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:13.898674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baec7ea-4afa-4e48-a240-027f9dce5735 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:13.935204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d301f72-91ea-4851-9e3d-e1c685678773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:13.935887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301f72-91ea-4851-9e3d-e1c685678773 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:13.936163 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301f72-91ea-4851-9e3d-e1c685678773 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:13.936442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301f72-91ea-4851-9e3d-e1c685678773 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:13.937175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d301f72-91ea-4851-9e3d-e1c685678773 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:13.937478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d301f72-91ea-4851-9e3d-e1c685678773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:13.938923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d301f72-91ea-4851-9e3d-e1c685678773 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:13.948989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 73/141] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:13.954636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876696e-400c-4788-9368-2c4f7bad06b4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:14.064277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876696e-400c-4788-9368-2c4f7bad06b4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:14.100515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3220c-5c90-48fd-a784-bf0f97941269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:14.101143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3220c-5c90-48fd-a784-bf0f97941269 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:14.101417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3220c-5c90-48fd-a784-bf0f97941269 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:14.101686 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3220c-5c90-48fd-a784-bf0f97941269 None admin] PATH_INFO: `/v3/users/2ff7daf364734feb8aab110b9b2ef63b` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:14.102858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3220c-5c90-48fd-a784-bf0f97941269 None admin] RBAC: Authorizing `identity:get_user(user_id=2ff7daf364734feb8aab110b9b2ef63b)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:14.104400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3220c-5c90-48fd-a784-bf0f97941269 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:14.106648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 69/142] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:13 2022] GET /identity/v3/users/2ff7daf364734feb8aab110b9b2ef63b => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:14.112439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d301f72-91ea-4851-9e3d-e1c685678773 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:14.198017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9760f9c-4c19-4311-ba18-3d01187fc526 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:14.198640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9760f9c-4c19-4311-ba18-3d01187fc526 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:14.198640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9760f9c-4c19-4311-ba18-3d01187fc526 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:14.199947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 70/143] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:14.212913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:14.216701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af541b32-06b6-4b79-8fe9-1d1cadc38c4b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:14.216984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af541b32-06b6-4b79-8fe9-1d1cadc38c4b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:14.217253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af541b32-06b6-4b79-8fe9-1d1cadc38c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:14.244001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-af541b32-06b6-4b79-8fe9-1d1cadc38c4b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:14.248457 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-af541b32-06b6-4b79-8fe9-1d1cadc38c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:14.272812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d301f72-91ea-4851-9e3d-e1c685678773 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:14.337206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b363c3-bdb4-490d-b808-3318a833f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:14.337206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b363c3-bdb4-490d-b808-3318a833f89e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:14.337206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b363c3-bdb4-490d-b808-3318a833f89e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:14.337206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b363c3-bdb4-490d-b808-3318a833f89e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:14.363909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:14.379647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 71/144] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:14.410059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af541b32-06b6-4b79-8fe9-1d1cadc38c4b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:14.456495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b363c3-bdb4-490d-b808-3318a833f89e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:14.457945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b363c3-bdb4-490d-b808-3318a833f89e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:14.541383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:14.568601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 74/145] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:14.574860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b363c3-bdb4-490d-b808-3318a833f89e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:14.622725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af541b32-06b6-4b79-8fe9-1d1cadc38c4b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:14.679349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b363c3-bdb4-490d-b808-3318a833f89e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:14.712849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ff2c7-cc95-4925-8444-133edcf74c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:14.713823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff2c7-cc95-4925-8444-133edcf74c3d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:14.714315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff2c7-cc95-4925-8444-133edcf74c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:14.714779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff2c7-cc95-4925-8444-133edcf74c3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:14.718517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5170c2-2f27-4081-89c6-7ca68d2c0836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:14.719267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5170c2-2f27-4081-89c6-7ca68d2c0836 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:14.719546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5170c2-2f27-4081-89c6-7ca68d2c0836 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:14.719816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5170c2-2f27-4081-89c6-7ca68d2c0836 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:14.721054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5170c2-2f27-4081-89c6-7ca68d2c0836 None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:14.722648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5170c2-2f27-4081-89c6-7ca68d2c0836 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:14.724468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 75/146] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:14 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:14.730635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5170c2-2f27-4081-89c6-7ca68d2c0836 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:14.744011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ff2c7-cc95-4925-8444-133edcf74c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-844ff2c7-cc95-4925-8444-133edcf74c3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:14.748951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:14.749768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:14.750516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:14.753003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 72/147] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:14.758583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ff2c7-cc95-4925-8444-133edcf74c3d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:14.778944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:14.838313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5170c2-2f27-4081-89c6-7ca68d2c0836 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:14.874622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475a9ea-59d4-485b-a85a-888134a75c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:14.875272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475a9ea-59d4-485b-a85a-888134a75c10 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:14.875548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475a9ea-59d4-485b-a85a-888134a75c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:14.875818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475a9ea-59d4-485b-a85a-888134a75c10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:14.876649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475a9ea-59d4-485b-a85a-888134a75c10 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5, user.id=2ff7daf364734feb8aab110b9b2ef63b) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:14.876954 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475a9ea-59d4-485b-a85a-888134a75c10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:14.878477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475a9ea-59d4-485b-a85a-888134a75c10 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:14.884532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:14.885656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 76/148] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:14 2022] GET /identity/v3/role_assignments?user.id=2ff7daf364734feb8aab110b9b2ef63b&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=8417343e9ef64991bcf51bb9692da102 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:14.984963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ff2c7-cc95-4925-8444-133edcf74c3d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:15.051316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a3d4-1d0b-4000-a502-a2b4b5fc0bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:15.052642 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a3d4-1d0b-4000-a502-a2b4b5fc0bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:15.053345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a3d4-1d0b-4000-a502-a2b4b5fc0bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:15.054039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a3d4-1d0b-4000-a502-a2b4b5fc0bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:15.055880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a3d4-1d0b-4000-a502-a2b4b5fc0bd2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:15.056404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a3d4-1d0b-4000-a502-a2b4b5fc0bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:15.058019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a3d4-1d0b-4000-a502-a2b4b5fc0bd2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:15.067501 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 73/149] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:15.073222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475a9ea-59d4-485b-a85a-888134a75c10 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:15.215276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475a9ea-59d4-485b-a85a-888134a75c10 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:15.224445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:15.252270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6075a-2100-47af-a62c-56c7336c0578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:15.252823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6075a-2100-47af-a62c-56c7336c0578 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:15.253100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6075a-2100-47af-a62c-56c7336c0578 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:15.253369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6075a-2100-47af-a62c-56c7336c0578 None admin] PATH_INFO: `/v3/users/acbec83fb2f04975b8e667d4223dfe33` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:15.254485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6075a-2100-47af-a62c-56c7336c0578 None admin] RBAC: Authorizing `identity:get_user(user_id=acbec83fb2f04975b8e667d4223dfe33)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:15.256036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6075a-2100-47af-a62c-56c7336c0578 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:15.257613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 77/150] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:15 2022] GET /identity/v3/users/acbec83fb2f04975b8e667d4223dfe33 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:15.263379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a3d4-1d0b-4000-a502-a2b4b5fc0bd2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:15.382134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a3d4-1d0b-4000-a502-a2b4b5fc0bd2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:15.394922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:15.422694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ebe2b-cdb5-4ba9-9a4c-7e6c8cdb7344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:15.423337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ebe2b-cdb5-4ba9-9a4c-7e6c8cdb7344 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:15.423620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ebe2b-cdb5-4ba9-9a4c-7e6c8cdb7344 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:15.423884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ebe2b-cdb5-4ba9-9a4c-7e6c8cdb7344 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:15.495394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ebe2b-cdb5-4ba9-9a4c-7e6c8cdb7344 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:15.496851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ebe2b-cdb5-4ba9-9a4c-7e6c8cdb7344 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:15.609892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 74/151] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:15.616340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6075a-2100-47af-a62c-56c7336c0578 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:15.747764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6075a-2100-47af-a62c-56c7336c0578 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:15.767039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117cec9-53ca-4ddc-a8c3-8a224658a421 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:15.767389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117cec9-53ca-4ddc-a8c3-8a224658a421 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:15.767612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117cec9-53ca-4ddc-a8c3-8a224658a421 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:15.769084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 75/152] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:50:15.782255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef6875-11ee-4580-914f-0e9d232bfda7 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:15.782596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef6875-11ee-4580-914f-0e9d232bfda7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:15.782812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef6875-11ee-4580-914f-0e9d232bfda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:15.802350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa146b-b95a-47d2-878e-526e8e8e2e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:15.803010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa146b-b95a-47d2-878e-526e8e8e2e6c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:15.803296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa146b-b95a-47d2-878e-526e8e8e2e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:15.803405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d0ef6875-11ee-4580-914f-0e9d232bfda7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:15.803566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa146b-b95a-47d2-878e-526e8e8e2e6c None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:15.804574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d0ef6875-11ee-4580-914f-0e9d232bfda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:15.807642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa146b-b95a-47d2-878e-526e8e8e2e6c None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:15.809116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa146b-b95a-47d2-878e-526e8e8e2e6c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:15.809273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:15.810943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 78/153] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:15 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:15.817633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa146b-b95a-47d2-878e-526e8e8e2e6c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:15.905000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:15.911741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 76/154] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:15.913088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3502a4-7f5a-40cd-a79e-c1eb96e60231 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:15.915067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3502a4-7f5a-40cd-a79e-c1eb96e60231 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:15.915067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3502a4-7f5a-40cd-a79e-c1eb96e60231 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:15.915067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 77/155] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:15.925007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3f04-28ef-4b64-9caa-e92e1cf84bd0 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:15.925562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3f04-28ef-4b64-9caa-e92e1cf84bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:15.925837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3f04-28ef-4b64-9caa-e92e1cf84bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:15.949080 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-43bd3f04-28ef-4b64-9caa-e92e1cf84bd0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:15.950282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-43bd3f04-28ef-4b64-9caa-e92e1cf84bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:15.996778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa146b-b95a-47d2-878e-526e8e8e2e6c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:16.030549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 78/156] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:16.032855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afdb16-4d5d-4e42-86c4-5c9a988a6451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:16.033458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afdb16-4d5d-4e42-86c4-5c9a988a6451 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.033730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afdb16-4d5d-4e42-86c4-5c9a988a6451 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.033995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afdb16-4d5d-4e42-86c4-5c9a988a6451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.034713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afdb16-4d5d-4e42-86c4-5c9a988a6451 None admin] RBAC: Adding query filter params (role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5, user.id=acbec83fb2f04975b8e667d4223dfe33) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:16.035054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afdb16-4d5d-4e42-86c4-5c9a988a6451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:16.036055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3f04-28ef-4b64-9caa-e92e1cf84bd0 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:16.036494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afdb16-4d5d-4e42-86c4-5c9a988a6451 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:16.043448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 79/157] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:15 2022] GET /identity/v3/role_assignments?user.id=acbec83fb2f04975b8e667d4223dfe33&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:16.052625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f2e70-f30e-4942-9adb-3d2796439ad8 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.052983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f2e70-f30e-4942-9adb-3d2796439ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.053168 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f2e70-f30e-4942-9adb-3d2796439ad8 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.059213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 80/158] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:16.059213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f2e70-f30e-4942-9adb-3d2796439ad8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:16.123656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:16.160618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3f04-28ef-4b64-9caa-e92e1cf84bd0 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:16.191844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f2e70-f30e-4942-9adb-3d2796439ad8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:16.200041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd830d7-8fe2-4cbb-880f-33ef398e2bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:16.200687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd830d7-8fe2-4cbb-880f-33ef398e2bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.200960 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd830d7-8fe2-4cbb-880f-33ef398e2bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.201227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd830d7-8fe2-4cbb-880f-33ef398e2bb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.202541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd830d7-8fe2-4cbb-880f-33ef398e2bb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:16.204041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd830d7-8fe2-4cbb-880f-33ef398e2bb7 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:16.205631 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 79/159] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:16.206773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ae8f-6568-4b71-8629-502dcacfe69c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.207082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ae8f-6568-4b71-8629-502dcacfe69c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.207295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:16.207404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ae8f-6568-4b71-8629-502dcacfe69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.228802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-b188ae8f-6568-4b71-8629-502dcacfe69c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:16.230001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b188ae8f-6568-4b71-8629-502dcacfe69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:16.237123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4be428-6a5f-4ab1-9801-0bf89c433659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:16.237760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4be428-6a5f-4ab1-9801-0bf89c433659 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.238042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4be428-6a5f-4ab1-9801-0bf89c433659 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.238315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4be428-6a5f-4ab1-9801-0bf89c433659 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.251491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4be428-6a5f-4ab1-9801-0bf89c433659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-5a4be428-6a5f-4ab1-9801-0bf89c433659 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:16.256230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:16.257072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:16.257841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 81/160] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:16.258504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4be428-6a5f-4ab1-9801-0bf89c433659 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:16.320529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 80/161] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:16.324210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188ae8f-6568-4b71-8629-502dcacfe69c None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:16.379197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4be428-6a5f-4ab1-9801-0bf89c433659 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:16.402270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:16.417010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a3ce6-5b2a-47d6-8bce-808c3438a5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:16.417631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a3ce6-5b2a-47d6-8bce-808c3438a5a8 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.418189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a3ce6-5b2a-47d6-8bce-808c3438a5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.418489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a3ce6-5b2a-47d6-8bce-808c3438a5a8 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.419169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a3ce6-5b2a-47d6-8bce-808c3438a5a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:16.420617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a3ce6-5b2a-47d6-8bce-808c3438a5a8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:16.443181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188ae8f-6568-4b71-8629-502dcacfe69c None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:16.444800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 82/162] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:16 2022] POST /identity/v3/users => generated 283 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:16.446922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a3ce6-5b2a-47d6-8bce-808c3438a5a8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:16.489230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:16.491250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b005b-2f0a-461b-878d-57caa9f2c705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:16.493372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b005b-2f0a-461b-878d-57caa9f2c705 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.494270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b005b-2f0a-461b-878d-57caa9f2c705 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.495068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b005b-2f0a-461b-878d-57caa9f2c705 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.496017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b005b-2f0a-461b-878d-57caa9f2c705 None admin] RBAC: Adding query filter params (name=member) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:16.496571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b005b-2f0a-461b-878d-57caa9f2c705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:16.498911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b005b-2f0a-461b-878d-57caa9f2c705 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:16.512889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 81/163] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:16.518356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b005b-2f0a-461b-878d-57caa9f2c705 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:16.591092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a3ce6-5b2a-47d6-8bce-808c3438a5a8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:16.628318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235bde1-55bc-44b7-9520-85e8c17db5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:16.628952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235bde1-55bc-44b7-9520-85e8c17db5cc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.629232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235bde1-55bc-44b7-9520-85e8c17db5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.629501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235bde1-55bc-44b7-9520-85e8c17db5cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.639853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b005b-2f0a-461b-878d-57caa9f2c705 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:16.640081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235bde1-55bc-44b7-9520-85e8c17db5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-8235bde1-55bc-44b7-9520-85e8c17db5cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:16.645255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:16.646143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:16.647465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 83/164] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:16.648332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529db413-6ab7-4154-9264-c3d4f8d81fcf None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.648332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529db413-6ab7-4154-9264-c3d4f8d81fcf None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.648332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529db413-6ab7-4154-9264-c3d4f8d81fcf None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.649806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 84/165] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:16.655632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef063e-aeee-4449-bba7-2c7973302543 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.655916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef063e-aeee-4449-bba7-2c7973302543 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.656193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef063e-aeee-4449-bba7-2c7973302543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.672998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:16.675869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-65ef063e-aeee-4449-bba7-2c7973302543 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:16.678166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-65ef063e-aeee-4449-bba7-2c7973302543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:16.682809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9778ad-0134-4916-a13f-91083d238eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:16.683462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9778ad-0134-4916-a13f-91083d238eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.683738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9778ad-0134-4916-a13f-91083d238eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.684003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9778ad-0134-4916-a13f-91083d238eb4 None admin] PATH_INFO: `/v3/users/acbec83fb2f04975b8e667d4223dfe33` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.686983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9778ad-0134-4916-a13f-91083d238eb4 None admin] RBAC: Authorizing `identity:get_user(user_id=acbec83fb2f04975b8e667d4223dfe33)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:16.686983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9778ad-0134-4916-a13f-91083d238eb4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:16.688700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 82/166] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:16 2022] GET /identity/v3/users/acbec83fb2f04975b8e667d4223dfe33 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:16.689247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9778ad-0134-4916-a13f-91083d238eb4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:16.771713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 85/167] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:16.772643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef063e-aeee-4449-bba7-2c7973302543 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:16.788746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:16.819919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9778ad-0134-4916-a13f-91083d238eb4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:16.862322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbc69c-ac8a-4f2e-905d-d9d06df37a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:16.863990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbc69c-ac8a-4f2e-905d-d9d06df37a5b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.864266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbc69c-ac8a-4f2e-905d-d9d06df37a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.864538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbc69c-ac8a-4f2e-905d-d9d06df37a5b None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.865211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbc69c-ac8a-4f2e-905d-d9d06df37a5b None admin] RBAC: Adding query filter params (name=member) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:16.865512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbc69c-ac8a-4f2e-905d-d9d06df37a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:16.866912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbc69c-ac8a-4f2e-905d-d9d06df37a5b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:16.877698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 83/168] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:16.878523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbc69c-ac8a-4f2e-905d-d9d06df37a5b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:16.895191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef063e-aeee-4449-bba7-2c7973302543 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:16.939402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086af0e-ceb9-4a65-a16c-33a2daf3cba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:16.939935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086af0e-ceb9-4a65-a16c-33a2daf3cba6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:16.940363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086af0e-ceb9-4a65-a16c-33a2daf3cba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:16.940635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086af0e-ceb9-4a65-a16c-33a2daf3cba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:16.948201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:16.999244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbc69c-ac8a-4f2e-905d-d9d06df37a5b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:17.028637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086af0e-ceb9-4a65-a16c-33a2daf3cba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:17.030123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086af0e-ceb9-4a65-a16c-33a2daf3cba6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:17.045998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dcf8f-1a65-49b1-a41c-dbc396261e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:17.047248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dcf8f-1a65-49b1-a41c-dbc396261e12 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:17.047925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dcf8f-1a65-49b1-a41c-dbc396261e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:17.048343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dcf8f-1a65-49b1-a41c-dbc396261e12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:17.059374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:17.062600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dcf8f-1a65-49b1-a41c-dbc396261e12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-469dcf8f-1a65-49b1-a41c-dbc396261e12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:17.069373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:17.070247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:17.071577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 84/169] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:17.072215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dcf8f-1a65-49b1-a41c-dbc396261e12 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:17.143199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 86/170] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:17.144319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086af0e-ceb9-4a65-a16c-33a2daf3cba6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:17.227972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dcf8f-1a65-49b1-a41c-dbc396261e12 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:17.267947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:17.306331 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6a8db-884d-425d-93d6-3f4b9eb8bd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:17.307098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6a8db-884d-425d-93d6-3f4b9eb8bd5a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:17.307378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6a8db-884d-425d-93d6-3f4b9eb8bd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:17.307648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6a8db-884d-425d-93d6-3f4b9eb8bd5a None admin] PATH_INFO: `/v3/users/acbec83fb2f04975b8e667d4223dfe33` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:17.308931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6a8db-884d-425d-93d6-3f4b9eb8bd5a None admin] RBAC: Authorizing `identity:get_user(user_id=acbec83fb2f04975b8e667d4223dfe33)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:17.310376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6a8db-884d-425d-93d6-3f4b9eb8bd5a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:17.310749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086af0e-ceb9-4a65-a16c-33a2daf3cba6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:17.312356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 85/171] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:17 2022] GET /identity/v3/users/acbec83fb2f04975b8e667d4223dfe33 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:17.313237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6a8db-884d-425d-93d6-3f4b9eb8bd5a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:17.327339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:17.363122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5fe72-2330-4c17-91a5-d8487ec41c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:17.363730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5fe72-2330-4c17-91a5-d8487ec41c6e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:17.364002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5fe72-2330-4c17-91a5-d8487ec41c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:17.364270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5fe72-2330-4c17-91a5-d8487ec41c6e None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:17.364933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5fe72-2330-4c17-91a5-d8487ec41c6e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:17.365230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5fe72-2330-4c17-91a5-d8487ec41c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:17.366592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5fe72-2330-4c17-91a5-d8487ec41c6e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:17.386859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 87/172] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:17.387669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5fe72-2330-4c17-91a5-d8487ec41c6e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:17.499633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5fe72-2330-4c17-91a5-d8487ec41c6e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:17.537629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ddd7a-63b5-4034-ae9d-56b42c14e1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:17.538298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ddd7a-63b5-4034-ae9d-56b42c14e1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:17.538573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ddd7a-63b5-4034-ae9d-56b42c14e1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:17.538867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ddd7a-63b5-4034-ae9d-56b42c14e1b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:17.541971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6a8db-884d-425d-93d6-3f4b9eb8bd5a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:17.590972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:17.611619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9bde0-83f4-47df-930a-bf06f0a0453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:17.612630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9bde0-83f4-47df-930a-bf06f0a0453c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:17.613105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9bde0-83f4-47df-930a-bf06f0a0453c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:17.613567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9bde0-83f4-47df-930a-bf06f0a0453c None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:17.618071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ddd7a-63b5-4034-ae9d-56b42c14e1b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:17.618071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ddd7a-63b5-4034-ae9d-56b42c14e1b1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:17.619124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9bde0-83f4-47df-930a-bf06f0a0453c None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:17.620845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9bde0-83f4-47df-930a-bf06f0a0453c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:17.623170 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 86/173] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:17 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:17.624191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9bde0-83f4-47df-930a-bf06f0a0453c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:17.720106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 88/174] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:17.721058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ddd7a-63b5-4034-ae9d-56b42c14e1b1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:17.745788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:17.813431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9bde0-83f4-47df-930a-bf06f0a0453c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:17.821218 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:17.848988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c33b9e-58bf-466d-8908-4048a1d9b71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:17.849621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c33b9e-58bf-466d-8908-4048a1d9b71d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:17.849899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c33b9e-58bf-466d-8908-4048a1d9b71d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:17.850169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c33b9e-58bf-466d-8908-4048a1d9b71d None admin] PATH_INFO: `/v3/users/6dcc22ec2d8e4a8d9dfc14e251b0c891` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:17.853180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c33b9e-58bf-466d-8908-4048a1d9b71d None admin] RBAC: Authorizing `identity:get_user(user_id=6dcc22ec2d8e4a8d9dfc14e251b0c891)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:17.854621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c33b9e-58bf-466d-8908-4048a1d9b71d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:17.857290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 87/175] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:17 2022] GET /identity/v3/users/6dcc22ec2d8e4a8d9dfc14e251b0c891 => generated 289 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:17.858865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c33b9e-58bf-466d-8908-4048a1d9b71d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:17.865961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ddd7a-63b5-4034-ae9d-56b42c14e1b1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:17.917809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336f4cc-0243-44ec-aeed-92e352f6080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:17.918702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f4cc-0243-44ec-aeed-92e352f6080e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:17.922978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f4cc-0243-44ec-aeed-92e352f6080e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:17.922978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f4cc-0243-44ec-aeed-92e352f6080e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:17.922978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336f4cc-0243-44ec-aeed-92e352f6080e None admin] RBAC: Adding query filter params (role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5, user.id=acbec83fb2f04975b8e667d4223dfe33) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:17.922978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336f4cc-0243-44ec-aeed-92e352f6080e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:17.922978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336f4cc-0243-44ec-aeed-92e352f6080e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:17.932135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 89/176] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:17 2022] GET /identity/v3/role_assignments?user.id=acbec83fb2f04975b8e667d4223dfe33&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:17.933154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336f4cc-0243-44ec-aeed-92e352f6080e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:18.088707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c33b9e-58bf-466d-8908-4048a1d9b71d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:18.119658 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336f4cc-0243-44ec-aeed-92e352f6080e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:18.142576 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:18.170976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab37a14-2cb7-4738-9c46-e106a06b1f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:18.172043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab37a14-2cb7-4738-9c46-e106a06b1f75 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:18.172564 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab37a14-2cb7-4738-9c46-e106a06b1f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:18.173032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab37a14-2cb7-4738-9c46-e106a06b1f75 None admin] PATH_INFO: `/v3/projects/60292cebe3104fc683cb7b1c8cff3f07` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:18.178256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab37a14-2cb7-4738-9c46-e106a06b1f75 None admin] RBAC: Authorizing `identity:get_project(project_id=60292cebe3104fc683cb7b1c8cff3f07)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:18.180140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab37a14-2cb7-4738-9c46-e106a06b1f75 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:18.182368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 88/177] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:17 2022] GET /identity/v3/projects/60292cebe3104fc683cb7b1c8cff3f07 => generated 312 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:18.187792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f7a88-6653-4293-a53e-28ddd96fac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:18.188470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f7a88-6653-4293-a53e-28ddd96fac1d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:18.188754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f7a88-6653-4293-a53e-28ddd96fac1d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:18.189033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f7a88-6653-4293-a53e-28ddd96fac1d None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:18.189287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab37a14-2cb7-4738-9c46-e106a06b1f75 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:18.190255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f7a88-6653-4293-a53e-28ddd96fac1d None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:18.191810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f7a88-6653-4293-a53e-28ddd96fac1d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:18.193617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 90/178] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:17 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:18.201984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f7a88-6653-4293-a53e-28ddd96fac1d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:18.226617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:18.390989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab37a14-2cb7-4738-9c46-e106a06b1f75 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:18.396575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f7a88-6653-4293-a53e-28ddd96fac1d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:18.454717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c564a-9425-42a6-9a54-e4cd95f237d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:18.456581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c564a-9425-42a6-9a54-e4cd95f237d3 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:18.457356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c564a-9425-42a6-9a54-e4cd95f237d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:18.458327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c564a-9425-42a6-9a54-e4cd95f237d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:18.459740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c564a-9425-42a6-9a54-e4cd95f237d3 None admin] RBAC: Adding query filter params (role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=60292cebe3104fc683cb7b1c8cff3f07, user.id=acbec83fb2f04975b8e667d4223dfe33) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:18.460145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e3eec-27c3-4fd7-9462-9afd5a2b4184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:18.461018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e3eec-27c3-4fd7-9462-9afd5a2b4184 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:18.462067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c564a-9425-42a6-9a54-e4cd95f237d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:18.462526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e3eec-27c3-4fd7-9462-9afd5a2b4184 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:18.463002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e3eec-27c3-4fd7-9462-9afd5a2b4184 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/users/6dcc22ec2d8e4a8d9dfc14e251b0c891/roles/053f81c391444db1b6409c4eb953dc35` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:18.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c564a-9425-42a6-9a54-e4cd95f237d3 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:18.466800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e3eec-27c3-4fd7-9462-9afd5a2b4184 None admin] RBAC: Authorizing `identity:create_grant(project_id=02ab39fdbee942d8bd432a27ca1d13c5, user_id=6dcc22ec2d8e4a8d9dfc14e251b0c891, role_id=053f81c391444db1b6409c4eb953dc35)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:18.470235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e3eec-27c3-4fd7-9462-9afd5a2b4184 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:18.476440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 89/179] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:18 2022] GET /identity/v3/role_assignments?user.id=acbec83fb2f04975b8e667d4223dfe33&scope.project.id=60292cebe3104fc683cb7b1c8cff3f07&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:18.511875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 91/180] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:50:18 2022] PUT /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/users/6dcc22ec2d8e4a8d9dfc14e251b0c891/roles/053f81c391444db1b6409c4eb953dc35 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:18.950643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fe93e-d165-41c7-9193-47dee16a4bc2 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:18.951201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fe93e-d165-41c7-9193-47dee16a4bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:18.951201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fe93e-d165-41c7-9193-47dee16a4bc2 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:18.952694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 90/181] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:18.957966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59b27d-7720-4a04-9262-f18975282efd None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:18.958245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59b27d-7720-4a04-9262-f18975282efd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:18.958514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59b27d-7720-4a04-9262-f18975282efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:18.977748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ef59b27d-7720-4a04-9262-f18975282efd None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:18.978910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ef59b27d-7720-4a04-9262-f18975282efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:19.042485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:19.048426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 92/182] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:19.073308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fe93e-d165-41c7-9193-47dee16a4bc2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:19.131886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:19.177834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fe93e-d165-41c7-9193-47dee16a4bc2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:19.191725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9556627-619b-4995-9ac5-b0dd69dd719f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.191725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9556627-619b-4995-9ac5-b0dd69dd719f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.191725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9556627-619b-4995-9ac5-b0dd69dd719f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.191725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 93/183] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:50:19.197503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b0d8e-9910-4a14-bf81-89d9d428f9da None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.198003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b0d8e-9910-4a14-bf81-89d9d428f9da None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.198469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b0d8e-9910-4a14-bf81-89d9d428f9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.215334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fe7b7-cc13-451d-9bdf-eb5b5241ce69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:19.215891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fe7b7-cc13-451d-9bdf-eb5b5241ce69 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.216211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fe7b7-cc13-451d-9bdf-eb5b5241ce69 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.216485 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fe7b7-cc13-451d-9bdf-eb5b5241ce69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.218111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ad2b0d8e-9910-4a14-bf81-89d9d428f9da None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:19.219555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ad2b0d8e-9910-4a14-bf81-89d9d428f9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:19.233296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fe7b7-cc13-451d-9bdf-eb5b5241ce69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-fe1fe7b7-cc13-451d-9bdf-eb5b5241ce69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:19.240116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:19.240996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:19.241773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 91/184] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:50:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:19.248953 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fe7b7-cc13-451d-9bdf-eb5b5241ce69 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:19.297253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 94/185] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:19.320270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b0d8e-9910-4a14-bf81-89d9d428f9da None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:19.367939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fe7b7-cc13-451d-9bdf-eb5b5241ce69 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:19.400963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:19.412990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea909e4-1530-4793-9ed3-214d561b845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:19.413715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea909e4-1530-4793-9ed3-214d561b845e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.413993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea909e4-1530-4793-9ed3-214d561b845e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.414317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea909e4-1530-4793-9ed3-214d561b845e None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.415048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea909e4-1530-4793-9ed3-214d561b845e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:19.415607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea909e4-1530-4793-9ed3-214d561b845e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:19.417436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea909e4-1530-4793-9ed3-214d561b845e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:19.428985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 92/186] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:19.434706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea909e4-1530-4793-9ed3-214d561b845e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:19.441500 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b0d8e-9910-4a14-bf81-89d9d428f9da None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:19.480494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:19.492154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d518da-324e-44b6-a8c6-ed112c3e491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:19.493646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d518da-324e-44b6-a8c6-ed112c3e491a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.494471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d518da-324e-44b6-a8c6-ed112c3e491a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.494936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d518da-324e-44b6-a8c6-ed112c3e491a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.510193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d518da-324e-44b6-a8c6-ed112c3e491a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-82d518da-324e-44b6-a8c6-ed112c3e491a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:19.515560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:19.516457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:19.517667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 95/187] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:19.518506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791a181-a404-491c-bf3e-1ed3c98a40cb None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.518744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791a181-a404-491c-bf3e-1ed3c98a40cb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.519042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791a181-a404-491c-bf3e-1ed3c98a40cb None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.520409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 96/188] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:19.522717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791a181-a404-491c-bf3e-1ed3c98a40cb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:19.545569 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea909e4-1530-4793-9ed3-214d561b845e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:19.581551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121b8eb-7b0d-4b8c-a177-70b3f3adae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:19.582497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121b8eb-7b0d-4b8c-a177-70b3f3adae2c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.582846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121b8eb-7b0d-4b8c-a177-70b3f3adae2c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.583219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121b8eb-7b0d-4b8c-a177-70b3f3adae2c None admin] PATH_INFO: `/v3/users/b118ad55fa4941a39e84abb92c57461a` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.597531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f121b8eb-7b0d-4b8c-a177-70b3f3adae2c None admin] RBAC: Authorizing `identity:get_user(user_id=b118ad55fa4941a39e84abb92c57461a)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:19.599025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f121b8eb-7b0d-4b8c-a177-70b3f3adae2c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:19.603154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 93/189] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:19 2022] GET /identity/v3/users/b118ad55fa4941a39e84abb92c57461a => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:19.604262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119b3fb-9a1d-4188-85e6-f437f38d090c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.604545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119b3fb-9a1d-4188-85e6-f437f38d090c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.605632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119b3fb-9a1d-4188-85e6-f437f38d090c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.624593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-5119b3fb-9a1d-4188-85e6-f437f38d090c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:19.625767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5119b3fb-9a1d-4188-85e6-f437f38d090c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:19.629438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791a181-a404-491c-bf3e-1ed3c98a40cb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:19.655401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:19.665929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb5ee4-c033-45b8-87cc-b3b208147576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:19.666586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb5ee4-c033-45b8-87cc-b3b208147576 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.666871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb5ee4-c033-45b8-87cc-b3b208147576 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.667176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb5ee4-c033-45b8-87cc-b3b208147576 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.667841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb5ee4-c033-45b8-87cc-b3b208147576 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:19.668141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb5ee4-c033-45b8-87cc-b3b208147576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:19.669630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb5ee4-c033-45b8-87cc-b3b208147576 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:19.678513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 97/190] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:19.683882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb5ee4-c033-45b8-87cc-b3b208147576 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:19.695721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 94/191] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:19.703819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119b3fb-9a1d-4188-85e6-f437f38d090c None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:19.736372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:19.787558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb5ee4-c033-45b8-87cc-b3b208147576 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:19.811269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119b3fb-9a1d-4188-85e6-f437f38d090c None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:19.823201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10811d4e-b9a8-402a-8192-4ca5cc983577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:19.823814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10811d4e-b9a8-402a-8192-4ca5cc983577 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.824095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10811d4e-b9a8-402a-8192-4ca5cc983577 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.824366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10811d4e-b9a8-402a-8192-4ca5cc983577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.848604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591f6f5-d0dd-4678-986f-abe2e848a787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:19.849582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591f6f5-d0dd-4678-986f-abe2e848a787 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:19.850048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591f6f5-d0dd-4678-986f-abe2e848a787 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:19.850520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591f6f5-d0dd-4678-986f-abe2e848a787 None admin] PATH_INFO: `/v3/users/b118ad55fa4941a39e84abb92c57461a` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:19.851759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591f6f5-d0dd-4678-986f-abe2e848a787 None admin] RBAC: Authorizing `identity:get_user(user_id=b118ad55fa4941a39e84abb92c57461a)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:19.853597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591f6f5-d0dd-4678-986f-abe2e848a787 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:19.859187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 95/192] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:19 2022] GET /identity/v3/users/b118ad55fa4941a39e84abb92c57461a => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:19.859187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591f6f5-d0dd-4678-986f-abe2e848a787 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:19.898223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:19.898590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10811d4e-b9a8-402a-8192-4ca5cc983577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:19.900202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10811d4e-b9a8-402a-8192-4ca5cc983577 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:19.949797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:19.969829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591f6f5-d0dd-4678-986f-abe2e848a787 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:20.007118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 98/193] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:20.007118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10811d4e-b9a8-402a-8192-4ca5cc983577 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:20.010601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0987c5d-639a-4a57-8f60-83a6fbc21858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:20.011264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0987c5d-639a-4a57-8f60-83a6fbc21858 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:20.011538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0987c5d-639a-4a57-8f60-83a6fbc21858 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:20.011802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0987c5d-639a-4a57-8f60-83a6fbc21858 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:20.012888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0987c5d-639a-4a57-8f60-83a6fbc21858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:20.014333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0987c5d-639a-4a57-8f60-83a6fbc21858 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:20.016010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 96/194] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:20.020060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0987c5d-639a-4a57-8f60-83a6fbc21858 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:20.107342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10811d4e-b9a8-402a-8192-4ca5cc983577 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:20.121375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0987c5d-639a-4a57-8f60-83a6fbc21858 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:20.123450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:20.146362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9c25d-83fa-423b-bfcd-9df5b7f6d9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:20.147020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9c25d-83fa-423b-bfcd-9df5b7f6d9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:20.147299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9c25d-83fa-423b-bfcd-9df5b7f6d9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:20.147756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9c25d-83fa-423b-bfcd-9df5b7f6d9e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:20.160179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc369db6-05ba-4546-a02d-497665441fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:20.160813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc369db6-05ba-4546-a02d-497665441fbb None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:20.161093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc369db6-05ba-4546-a02d-497665441fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:20.161368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc369db6-05ba-4546-a02d-497665441fbb None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:20.162492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc369db6-05ba-4546-a02d-497665441fbb None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:20.163981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc369db6-05ba-4546-a02d-497665441fbb None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:20.166988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 97/195] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:20 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:20.166988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc369db6-05ba-4546-a02d-497665441fbb None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:20.172245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:20.224265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9c25d-83fa-423b-bfcd-9df5b7f6d9e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:20.225730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9c25d-83fa-423b-bfcd-9df5b7f6d9e5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:20.269315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc369db6-05ba-4546-a02d-497665441fbb None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:20.303815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc2594-e0b1-4df1-a04a-18e8a8791416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:20.304424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc2594-e0b1-4df1-a04a-18e8a8791416 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:20.304700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc2594-e0b1-4df1-a04a-18e8a8791416 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:20.305617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc2594-e0b1-4df1-a04a-18e8a8791416 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:20.306252 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc2594-e0b1-4df1-a04a-18e8a8791416 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:20.309621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc2594-e0b1-4df1-a04a-18e8a8791416 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:20.338214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 98/196] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:20 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:20.339462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc2594-e0b1-4df1-a04a-18e8a8791416 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:20.350574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 99/197] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:20.357693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9c25d-83fa-423b-bfcd-9df5b7f6d9e5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:20.382405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:20.436346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:20.488246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc2594-e0b1-4df1-a04a-18e8a8791416 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:20.509843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9c25d-83fa-423b-bfcd-9df5b7f6d9e5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:20.537902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029439a5-72b6-4e1a-aee5-7c1cc445ed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:20.538712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029439a5-72b6-4e1a-aee5-7c1cc445ed15 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:20.539068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029439a5-72b6-4e1a-aee5-7c1cc445ed15 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:20.539396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029439a5-72b6-4e1a-aee5-7c1cc445ed15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:20.540615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029439a5-72b6-4e1a-aee5-7c1cc445ed15 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=d254037584f44b54b09c2b3e76807918, user.id=b118ad55fa4941a39e84abb92c57461a) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:20.541081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029439a5-72b6-4e1a-aee5-7c1cc445ed15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:20.543182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029439a5-72b6-4e1a-aee5-7c1cc445ed15 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:20.554004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e39d1b-7265-4b0d-96c4-efd26e746b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:20.555188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e39d1b-7265-4b0d-96c4-efd26e746b84 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:20.555469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e39d1b-7265-4b0d-96c4-efd26e746b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:20.555741 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e39d1b-7265-4b0d-96c4-efd26e746b84 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:20.556970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e39d1b-7265-4b0d-96c4-efd26e746b84 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:20.558462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e39d1b-7265-4b0d-96c4-efd26e746b84 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:20.559052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 99/198] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:20 2022] GET /identity/v3/role_assignments?user.id=b118ad55fa4941a39e84abb92c57461a&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=8417343e9ef64991bcf51bb9692da102 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:20.560376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 100/199] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:20 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:20.566544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029439a5-72b6-4e1a-aee5-7c1cc445ed15 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:20.681548 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:20.694619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029439a5-72b6-4e1a-aee5-7c1cc445ed15 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:20.734153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a164987-7314-4641-ae7f-5842a3fc37e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:20.734868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a164987-7314-4641-ae7f-5842a3fc37e9 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:20.735227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a164987-7314-4641-ae7f-5842a3fc37e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:20.735499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a164987-7314-4641-ae7f-5842a3fc37e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:20.736276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a164987-7314-4641-ae7f-5842a3fc37e9 None admin] RBAC: Adding query filter params (role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=d254037584f44b54b09c2b3e76807918, user.id=b118ad55fa4941a39e84abb92c57461a) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:20.736647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a164987-7314-4641-ae7f-5842a3fc37e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:20.738146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a164987-7314-4641-ae7f-5842a3fc37e9 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:20.744276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 100/200] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:20 2022] GET /identity/v3/role_assignments?user.id=b118ad55fa4941a39e84abb92c57461a&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:20.762817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a612fcf-b694-4cc2-893a-88267638cbfc None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:20.763144 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a612fcf-b694-4cc2-893a-88267638cbfc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:20.763396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a612fcf-b694-4cc2-893a-88267638cbfc None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:20.764828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 101/201] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:20.770991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e6928-bdef-4e5c-b33b-501eb7d0a4e3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:20.771258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e6928-bdef-4e5c-b33b-501eb7d0a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:20.771533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e6928-bdef-4e5c-b33b-501eb7d0a4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:20.791660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-506e6928-bdef-4e5c-b33b-501eb7d0a4e3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:20.792867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-506e6928-bdef-4e5c-b33b-501eb7d0a4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:20.880716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 101/202] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:20.904568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a612fcf-b694-4cc2-893a-88267638cbfc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:21.011348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a612fcf-b694-4cc2-893a-88267638cbfc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:21.035519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:21.057000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d9e27-1b3f-4ab3-bcb1-88188188309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:21.057941 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d9e27-1b3f-4ab3-bcb1-88188188309a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:21.058486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d9e27-1b3f-4ab3-bcb1-88188188309a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:21.058926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d9e27-1b3f-4ab3-bcb1-88188188309a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:21.073935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d9e27-1b3f-4ab3-bcb1-88188188309a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-721d9e27-1b3f-4ab3-bcb1-88188188309a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:21.079077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:21.079963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:21.080742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:21.082975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 102/203] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:21.089243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e6928-bdef-4e5c-b33b-501eb7d0a4e3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:21.192737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:21.213941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e6928-bdef-4e5c-b33b-501eb7d0a4e3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:21.258748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbaf02a-04fc-4357-9a15-85cc5f5d72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:21.259643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbaf02a-04fc-4357-9a15-85cc5f5d72d0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:21.260090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbaf02a-04fc-4357-9a15-85cc5f5d72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:21.260498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbaf02a-04fc-4357-9a15-85cc5f5d72d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:21.261291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbaf02a-04fc-4357-9a15-85cc5f5d72d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:21.261760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbaf02a-04fc-4357-9a15-85cc5f5d72d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:21.263343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbaf02a-04fc-4357-9a15-85cc5f5d72d0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:21.277648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 102/204] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:21.283453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d9e27-1b3f-4ab3-bcb1-88188188309a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:21.409175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d9e27-1b3f-4ab3-bcb1-88188188309a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:21.452243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aaf64-b968-4ca5-be83-4d336dfd2d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:21.452868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aaf64-b968-4ca5-be83-4d336dfd2d51 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:21.453144 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aaf64-b968-4ca5-be83-4d336dfd2d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:21.453463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aaf64-b968-4ca5-be83-4d336dfd2d51 None admin] PATH_INFO: `/v3/users/6dcc22ec2d8e4a8d9dfc14e251b0c891` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:21.454755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aaf64-b968-4ca5-be83-4d336dfd2d51 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcc22ec2d8e4a8d9dfc14e251b0c891)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:21.456261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aaf64-b968-4ca5-be83-4d336dfd2d51 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:21.458123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 103/205] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:21 2022] GET /identity/v3/users/6dcc22ec2d8e4a8d9dfc14e251b0c891 => generated 289 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:21.463693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbaf02a-04fc-4357-9a15-85cc5f5d72d0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:21.593588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbaf02a-04fc-4357-9a15-85cc5f5d72d0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:21.640394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc24e55-645d-4518-8c42-9f99ab20af0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:21.641001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc24e55-645d-4518-8c42-9f99ab20af0f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:21.641276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc24e55-645d-4518-8c42-9f99ab20af0f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:21.641544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc24e55-645d-4518-8c42-9f99ab20af0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:21.662365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:21.730856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc24e55-645d-4518-8c42-9f99ab20af0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:21.732338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc24e55-645d-4518-8c42-9f99ab20af0f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:21.845842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 103/206] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:21.851448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aaf64-b968-4ca5-be83-4d336dfd2d51 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:21.907007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:21.982180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aaf64-b968-4ca5-be83-4d336dfd2d51 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:22.027957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71bfb3-2d80-47f2-957e-8c3dab5c92d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:22.028598 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71bfb3-2d80-47f2-957e-8c3dab5c92d4 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:22.028876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71bfb3-2d80-47f2-957e-8c3dab5c92d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:22.029145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71bfb3-2d80-47f2-957e-8c3dab5c92d4 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:22.030869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71bfb3-2d80-47f2-957e-8c3dab5c92d4 None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:22.032365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71bfb3-2d80-47f2-957e-8c3dab5c92d4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:22.034071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 104/207] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:21 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:22.041081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc24e55-645d-4518-8c42-9f99ab20af0f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:22.114390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:22.194025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc24e55-645d-4518-8c42-9f99ab20af0f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:22.244183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615dfdf-d2c1-4baf-a1d9-bba273866c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:22.244708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615dfdf-d2c1-4baf-a1d9-bba273866c68 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:22.244985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615dfdf-d2c1-4baf-a1d9-bba273866c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:22.245250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615dfdf-d2c1-4baf-a1d9-bba273866c68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:22.245976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615dfdf-d2c1-4baf-a1d9-bba273866c68 None admin] RBAC: Adding query filter params (role.id=053f81c391444db1b6409c4eb953dc35, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5, user.id=6dcc22ec2d8e4a8d9dfc14e251b0c891) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:22.246283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615dfdf-d2c1-4baf-a1d9-bba273866c68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:22.247753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615dfdf-d2c1-4baf-a1d9-bba273866c68 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:22.261940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 104/208] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:22 2022] GET /identity/v3/role_assignments?user.id=6dcc22ec2d8e4a8d9dfc14e251b0c891&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=053f81c391444db1b6409c4eb953dc35 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:23.019515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de9e3a-1e5a-4627-9550-a88102d2bcb3 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.020014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de9e3a-1e5a-4627-9550-a88102d2bcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.020088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de9e3a-1e5a-4627-9550-a88102d2bcb3 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.021532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 105/209] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:23.027350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d2e1b-b781-4cec-9854-fde56a963397 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.027626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d2e1b-b781-4cec-9854-fde56a963397 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.027892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d2e1b-b781-4cec-9854-fde56a963397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.045910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-708d2e1b-b781-4cec-9854-fde56a963397 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:23.047101 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-708d2e1b-b781-4cec-9854-fde56a963397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:23.110669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc680b-eb1f-4a6d-9033-613e3ed5de25 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.110968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc680b-eb1f-4a6d-9033-613e3ed5de25 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.111244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc680b-eb1f-4a6d-9033-613e3ed5de25 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.112586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 106/210] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:50:23.118337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f7b7f-aa35-4732-b316-e910240c9064 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.118879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f7b7f-aa35-4732-b316-e910240c9064 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.119447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f7b7f-aa35-4732-b316-e910240c9064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.119881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 105/211] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:23.139179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ef9f7b7f-aa35-4732-b316-e910240c9064 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:23.140720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ef9f7b7f-aa35-4732-b316-e910240c9064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:23.143629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d2e1b-b781-4cec-9854-fde56a963397 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.190023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:23.210798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 107/212] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:23.215257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed929aa-64da-47ed-a000-41a89f731c4a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.215539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed929aa-64da-47ed-a000-41a89f731c4a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.215812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed929aa-64da-47ed-a000-41a89f731c4a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.217210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 108/213] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:23.218060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed04f3-fe1b-4595-abea-e83be6036881 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.218333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed04f3-fe1b-4595-abea-e83be6036881 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.218601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed04f3-fe1b-4595-abea-e83be6036881 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.220920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 109/214] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:23.222684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc579-6690-4002-a5d7-4e58276ab94c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.222684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc579-6690-4002-a5d7-4e58276ab94c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.222988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc579-6690-4002-a5d7-4e58276ab94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.241842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a62bc579-6690-4002-a5d7-4e58276ab94c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:23.243342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a62bc579-6690-4002-a5d7-4e58276ab94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:23.252582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d2e1b-b781-4cec-9854-fde56a963397 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:23.282649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:23.287795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ba0a6-522b-489a-9fa1-12faa694e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:23.288422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ba0a6-522b-489a-9fa1-12faa694e738 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.288694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ba0a6-522b-489a-9fa1-12faa694e738 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.288959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ba0a6-522b-489a-9fa1-12faa694e738 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.298692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ba0a6-522b-489a-9fa1-12faa694e738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-ad3ba0a6-522b-489a-9fa1-12faa694e738 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:23.303406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:23.304297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:23.305589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 106/215] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:23.306788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa711a7-cdb7-4bd3-9b9e-dbded76a6192 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.307100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa711a7-cdb7-4bd3-9b9e-dbded76a6192 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.307371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa711a7-cdb7-4bd3-9b9e-dbded76a6192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.322990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 110/216] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:23.322990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bc579-6690-4002-a5d7-4e58276ab94c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.324502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-afa711a7-cdb7-4bd3-9b9e-dbded76a6192 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:23.325676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-afa711a7-cdb7-4bd3-9b9e-dbded76a6192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:23.389850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 107/217] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:23.390681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa711a7-cdb7-4bd3-9b9e-dbded76a6192 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.424359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bc579-6690-4002-a5d7-4e58276ab94c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:23.424694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:23.457969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2999526-4b60-43b5-a1bb-a68b7293e3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:23.458589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2999526-4b60-43b5-a1bb-a68b7293e3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.458861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2999526-4b60-43b5-a1bb-a68b7293e3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.459161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2999526-4b60-43b5-a1bb-a68b7293e3d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.468859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2999526-4b60-43b5-a1bb-a68b7293e3d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-b2999526-4b60-43b5-a1bb-a68b7293e3d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:23.473437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:23.474381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:23.475198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 111/218] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:23.475920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2999526-4b60-43b5-a1bb-a68b7293e3d1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.496562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa711a7-cdb7-4bd3-9b9e-dbded76a6192 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:23.509204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:23.530375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd28ab-9a5d-46a3-ad7f-007d4ccdd42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:23.531003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd28ab-9a5d-46a3-ad7f-007d4ccdd42f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.531284 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd28ab-9a5d-46a3-ad7f-007d4ccdd42f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.531552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd28ab-9a5d-46a3-ad7f-007d4ccdd42f None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.532215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd28ab-9a5d-46a3-ad7f-007d4ccdd42f None admin] RBAC: Adding query filter params (name=member) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:23.532516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd28ab-9a5d-46a3-ad7f-007d4ccdd42f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:23.533939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd28ab-9a5d-46a3-ad7f-007d4ccdd42f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:23.546419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 108/219] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:23.546419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd28ab-9a5d-46a3-ad7f-007d4ccdd42f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.571817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2999526-4b60-43b5-a1bb-a68b7293e3d1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:23.604121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d76a7-a611-48d3-8cd1-3073c5391cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:23.604743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d76a7-a611-48d3-8cd1-3073c5391cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.605018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d76a7-a611-48d3-8cd1-3073c5391cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.605289 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d76a7-a611-48d3-8cd1-3073c5391cc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.606387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d76a7-a611-48d3-8cd1-3073c5391cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:23.607842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d76a7-a611-48d3-8cd1-3073c5391cc3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:23.609431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 112/220] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:23.610186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d76a7-a611-48d3-8cd1-3073c5391cc3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.629248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:23.638545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd28ab-9a5d-46a3-ad7f-007d4ccdd42f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:23.670350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137de79d-e27f-4857-98d6-d7904a8736d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:23.670994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137de79d-e27f-4857-98d6-d7904a8736d8 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.671276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137de79d-e27f-4857-98d6-d7904a8736d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.671545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137de79d-e27f-4857-98d6-d7904a8736d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.682128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137de79d-e27f-4857-98d6-d7904a8736d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-137de79d-e27f-4857-98d6-d7904a8736d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:23.686700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:23.687848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 109/221] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:23.688821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137de79d-e27f-4857-98d6-d7904a8736d8 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.711411 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d76a7-a611-48d3-8cd1-3073c5391cc3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:23.712911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:23.745831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f442230-4385-4a29-a709-44915cd1e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:23.746430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f442230-4385-4a29-a709-44915cd1e69b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.746708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f442230-4385-4a29-a709-44915cd1e69b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.747007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f442230-4385-4a29-a709-44915cd1e69b None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.747668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f442230-4385-4a29-a709-44915cd1e69b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:23.747969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f442230-4385-4a29-a709-44915cd1e69b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:23.749324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f442230-4385-4a29-a709-44915cd1e69b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:23.756984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 113/222] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:23.757737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f442230-4385-4a29-a709-44915cd1e69b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.793210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137de79d-e27f-4857-98d6-d7904a8736d8 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:23.824947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a2564-5d16-45db-8f24-f232496fc5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:23.825560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a2564-5d16-45db-8f24-f232496fc5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.825835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a2564-5d16-45db-8f24-f232496fc5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.826104 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a2564-5d16-45db-8f24-f232496fc5e1 None admin] PATH_INFO: `/v3/users/0da08bc23fe548e4a41f4979cc6c525c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.838841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a2564-5d16-45db-8f24-f232496fc5e1 None admin] RBAC: Authorizing `identity:get_user(user_id=0da08bc23fe548e4a41f4979cc6c525c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:23.840320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a2564-5d16-45db-8f24-f232496fc5e1 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:23.841887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 110/223] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:23 2022] GET /identity/v3/users/0da08bc23fe548e4a41f4979cc6c525c => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:23.843773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a2564-5d16-45db-8f24-f232496fc5e1 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.851563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f442230-4385-4a29-a709-44915cd1e69b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:23.855706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:23.884857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed79b62-b9c4-4b46-a1b1-6a5733ce34e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:23.885456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed79b62-b9c4-4b46-a1b1-6a5733ce34e5 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.885732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed79b62-b9c4-4b46-a1b1-6a5733ce34e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.886002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed79b62-b9c4-4b46-a1b1-6a5733ce34e5 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.886633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed79b62-b9c4-4b46-a1b1-6a5733ce34e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:23.888073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed79b62-b9c4-4b46-a1b1-6a5733ce34e5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:23.907873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 114/224] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:23 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:23.908672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed79b62-b9c4-4b46-a1b1-6a5733ce34e5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:23.933806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:23.956750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a2564-5d16-45db-8f24-f232496fc5e1 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:23.997371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf03635-e5eb-4126-82ab-f2297d582911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:23.997990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf03635-e5eb-4126-82ab-f2297d582911 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:23.998267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf03635-e5eb-4126-82ab-f2297d582911 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:23.998539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf03635-e5eb-4126-82ab-f2297d582911 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:23.999242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf03635-e5eb-4126-82ab-f2297d582911 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:23.999539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf03635-e5eb-4126-82ab-f2297d582911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.000948 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf03635-e5eb-4126-82ab-f2297d582911 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.009847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 111/225] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:24.010707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf03635-e5eb-4126-82ab-f2297d582911 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:24.030666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed79b62-b9c4-4b46-a1b1-6a5733ce34e5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:24.070589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881ac59-22ba-43b3-bd91-102112387f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:24.071189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881ac59-22ba-43b3-bd91-102112387f74 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:24.071468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881ac59-22ba-43b3-bd91-102112387f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:24.071741 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881ac59-22ba-43b3-bd91-102112387f74 None admin] PATH_INFO: `/v3/users/b118ad55fa4941a39e84abb92c57461a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:24.073367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881ac59-22ba-43b3-bd91-102112387f74 None admin] RBAC: Authorizing `identity:get_user(user_id=b118ad55fa4941a39e84abb92c57461a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.074798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881ac59-22ba-43b3-bd91-102112387f74 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.076736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 115/226] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:23 2022] GET /identity/v3/users/b118ad55fa4941a39e84abb92c57461a => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:24.082063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881ac59-22ba-43b3-bd91-102112387f74 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:24.090299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:24.126161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf03635-e5eb-4126-82ab-f2297d582911 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:24.165958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e422e-c39d-4949-8a60-565eba10ae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:24.166585 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e422e-c39d-4949-8a60-565eba10ae1f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:24.166881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e422e-c39d-4949-8a60-565eba10ae1f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:24.167215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e422e-c39d-4949-8a60-565eba10ae1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:24.195194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:24.215090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881ac59-22ba-43b3-bd91-102112387f74 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:24.257832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571d34-5705-49ef-a6bf-6eee8a4fd8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:24.258561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571d34-5705-49ef-a6bf-6eee8a4fd8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:24.258842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571d34-5705-49ef-a6bf-6eee8a4fd8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:24.259153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571d34-5705-49ef-a6bf-6eee8a4fd8c4 None admin] PATH_INFO: `/v3/users/b118ad55fa4941a39e84abb92c57461a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:24.260346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571d34-5705-49ef-a6bf-6eee8a4fd8c4 None admin] RBAC: Authorizing `identity:get_user(user_id=b118ad55fa4941a39e84abb92c57461a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.261797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571d34-5705-49ef-a6bf-6eee8a4fd8c4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.263223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e422e-c39d-4949-8a60-565eba10ae1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.263451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 116/227] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:24 2022] GET /identity/v3/users/b118ad55fa4941a39e84abb92c57461a => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:24.264735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e422e-c39d-4949-8a60-565eba10ae1f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.267714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571d34-5705-49ef-a6bf-6eee8a4fd8c4 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:24.354370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:24.387596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 112/228] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:24.392482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e422e-c39d-4949-8a60-565eba10ae1f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:24.410232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571d34-5705-49ef-a6bf-6eee8a4fd8c4 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:24.458056 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d7c86-b971-4d75-b7c8-4283b3f5ff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:24.459299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d7c86-b971-4d75-b7c8-4283b3f5ff20 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:24.460226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d7c86-b971-4d75-b7c8-4283b3f5ff20 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:24.460755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d7c86-b971-4d75-b7c8-4283b3f5ff20 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:24.464351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d7c86-b971-4d75-b7c8-4283b3f5ff20 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.466810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d7c86-b971-4d75-b7c8-4283b3f5ff20 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.469810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 117/229] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:24 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:24.470727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d7c86-b971-4d75-b7c8-4283b3f5ff20 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:24.503940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:24.506199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e422e-c39d-4949-8a60-565eba10ae1f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:24.550414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ed73b-6a42-4f2f-bcdd-63411068dd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:24.551891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ed73b-6a42-4f2f-bcdd-63411068dd48 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:24.552307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ed73b-6a42-4f2f-bcdd-63411068dd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:24.552720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ed73b-6a42-4f2f-bcdd-63411068dd48 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:24.554326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ed73b-6a42-4f2f-bcdd-63411068dd48 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.557251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ed73b-6a42-4f2f-bcdd-63411068dd48 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.559084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 113/230] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:24 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:24.559967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ed73b-6a42-4f2f-bcdd-63411068dd48 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:24.593489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d7c86-b971-4d75-b7c8-4283b3f5ff20 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:24.616136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:24.637651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f52ae-6840-42c5-b7db-a3993cac71cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:24.638492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f52ae-6840-42c5-b7db-a3993cac71cf None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:24.639175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f52ae-6840-42c5-b7db-a3993cac71cf None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:24.640061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f52ae-6840-42c5-b7db-a3993cac71cf None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:24.641704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f52ae-6840-42c5-b7db-a3993cac71cf None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.643310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f52ae-6840-42c5-b7db-a3993cac71cf None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.650975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 118/231] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:24 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:24.650975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f52ae-6840-42c5-b7db-a3993cac71cf None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:24.685244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ed73b-6a42-4f2f-bcdd-63411068dd48 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:24.729594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe88b4-0886-498d-b043-e5fefb3e41f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:24.730502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe88b4-0886-498d-b043-e5fefb3e41f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:24.730918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe88b4-0886-498d-b043-e5fefb3e41f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:24.731349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe88b4-0886-498d-b043-e5fefb3e41f4 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918/users/b118ad55fa4941a39e84abb92c57461a/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:24.734385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe88b4-0886-498d-b043-e5fefb3e41f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d254037584f44b54b09c2b3e76807918, user_id=b118ad55fa4941a39e84abb92c57461a, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.736424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe88b4-0886-498d-b043-e5fefb3e41f4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.755834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 114/232] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:50:24 2022] PUT /identity/v3/projects/d254037584f44b54b09c2b3e76807918/users/b118ad55fa4941a39e84abb92c57461a/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:24.756864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe88b4-0886-498d-b043-e5fefb3e41f4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:24.772839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:24.789036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f52ae-6840-42c5-b7db-a3993cac71cf None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:24.866070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8868f-79a0-4f27-a505-d171f49c9730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:24.868725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8868f-79a0-4f27-a505-d171f49c9730 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:24.869394 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8868f-79a0-4f27-a505-d171f49c9730 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:24.871285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8868f-79a0-4f27-a505-d171f49c9730 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918/users/b118ad55fa4941a39e84abb92c57461a/roles/02c7974bceec49ee8895dabf30cc8fff` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:24.874974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8868f-79a0-4f27-a505-d171f49c9730 None admin] RBAC: Authorizing `identity:create_grant(project_id=d254037584f44b54b09c2b3e76807918, user_id=b118ad55fa4941a39e84abb92c57461a, role_id=02c7974bceec49ee8895dabf30cc8fff)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.878304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8868f-79a0-4f27-a505-d171f49c9730 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.900871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:24.910059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 119/233] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:50:24 2022] PUT /identity/v3/projects/d254037584f44b54b09c2b3e76807918/users/b118ad55fa4941a39e84abb92c57461a/roles/02c7974bceec49ee8895dabf30cc8fff => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:24.911694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe88b4-0886-498d-b043-e5fefb3e41f4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:24.951194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550361a3-5b15-4e36-a835-061626b64220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:24.952104 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550361a3-5b15-4e36-a835-061626b64220 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:24.952523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550361a3-5b15-4e36-a835-061626b64220 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:24.952930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550361a3-5b15-4e36-a835-061626b64220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:24.953810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550361a3-5b15-4e36-a835-061626b64220 None admin] RBAC: Adding query filter params (role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=d254037584f44b54b09c2b3e76807918, user.id=0da08bc23fe548e4a41f4979cc6c525c) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:24.954255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550361a3-5b15-4e36-a835-061626b64220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:24.955919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550361a3-5b15-4e36-a835-061626b64220 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:24.963310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 115/234] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:24 2022] GET /identity/v3/role_assignments?user.id=0da08bc23fe548e4a41f4979cc6c525c&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:26.221502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eae945-f6c9-41a7-be73-d1c952d0d42b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:26.222013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eae945-f6c9-41a7-be73-d1c952d0d42b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:26.222119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eae945-f6c9-41a7-be73-d1c952d0d42b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:26.223692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 120/235] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:26.225356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9531b-a56f-4f7f-8657-9d1593f71d2c None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:26.227960 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9531b-a56f-4f7f-8657-9d1593f71d2c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:26.228433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9531b-a56f-4f7f-8657-9d1593f71d2c None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:26.229229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c6a9b-30d0-44a7-93db-04dcef5cb790 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:26.229511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c6a9b-30d0-44a7-93db-04dcef5cb790 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:26.229787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c6a9b-30d0-44a7-93db-04dcef5cb790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:26.230588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 116/236] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:26.235766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95306763-1800-4d72-9c5e-d6e9d29e4c9c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:26.236271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95306763-1800-4d72-9c5e-d6e9d29e4c9c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:26.236758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95306763-1800-4d72-9c5e-d6e9d29e4c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:26.255975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-295c6a9b-30d0-44a7-93db-04dcef5cb790 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:26.256626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-95306763-1800-4d72-9c5e-d6e9d29e4c9c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:26.257236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-295c6a9b-30d0-44a7-93db-04dcef5cb790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:26.258222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-95306763-1800-4d72-9c5e-d6e9d29e4c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:26.333192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 117/237] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:26.359679 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95306763-1800-4d72-9c5e-d6e9d29e4c9c None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:26.394615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 121/238] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:26.430536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c6a9b-30d0-44a7-93db-04dcef5cb790 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:26.482793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:26.489420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95306763-1800-4d72-9c5e-d6e9d29e4c9c None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:26.532156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952aec5-e6bc-4c4b-9799-6a42ff07f772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:26.533360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952aec5-e6bc-4c4b-9799-6a42ff07f772 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:26.534488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952aec5-e6bc-4c4b-9799-6a42ff07f772 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:26.535171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952aec5-e6bc-4c4b-9799-6a42ff07f772 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:26.537696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952aec5-e6bc-4c4b-9799-6a42ff07f772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:26.541005 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952aec5-e6bc-4c4b-9799-6a42ff07f772 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:26.543528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 118/239] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:26.549550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952aec5-e6bc-4c4b-9799-6a42ff07f772 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:26.603334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:26.636311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c6a9b-30d0-44a7-93db-04dcef5cb790 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:26.694250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdbae1-fe39-4c1f-bc7d-61cd4cbafb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:26.696571 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdbae1-fe39-4c1f-bc7d-61cd4cbafb13 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:26.697790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdbae1-fe39-4c1f-bc7d-61cd4cbafb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:26.700922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdbae1-fe39-4c1f-bc7d-61cd4cbafb13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:26.720160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdbae1-fe39-4c1f-bc7d-61cd4cbafb13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:26.722606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952aec5-e6bc-4c4b-9799-6a42ff07f772 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-07bdbae1-fe39-4c1f-bc7d-61cd4cbafb13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:26.729391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:26.730256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:26.731112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:26.738885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 122/240] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:26.747083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdbae1-fe39-4c1f-bc7d-61cd4cbafb13 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:26.791235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075896a6-33a6-42e0-9eb6-bffdcf0ae850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:26.791849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075896a6-33a6-42e0-9eb6-bffdcf0ae850 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:26.792121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075896a6-33a6-42e0-9eb6-bffdcf0ae850 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:26.792391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075896a6-33a6-42e0-9eb6-bffdcf0ae850 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:26.793033 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075896a6-33a6-42e0-9eb6-bffdcf0ae850 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:26.794461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075896a6-33a6-42e0-9eb6-bffdcf0ae850 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:26.831203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 119/241] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:26 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:26.892987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdbae1-fe39-4c1f-bc7d-61cd4cbafb13 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:26.943341 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0551d55-1707-460f-bcd9-30ba81fb96b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:26.944009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0551d55-1707-460f-bcd9-30ba81fb96b4 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:26.944294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0551d55-1707-460f-bcd9-30ba81fb96b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:26.944564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0551d55-1707-460f-bcd9-30ba81fb96b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:26.945239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0551d55-1707-460f-bcd9-30ba81fb96b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:26.945575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0551d55-1707-460f-bcd9-30ba81fb96b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:26.946963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0551d55-1707-460f-bcd9-30ba81fb96b4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:26.958902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 123/242] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:26.964867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075896a6-33a6-42e0-9eb6-bffdcf0ae850 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:26.986716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:27.096521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075896a6-33a6-42e0-9eb6-bffdcf0ae850 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:27.132474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b4d54-95b6-4cd0-bd38-12a2f72400da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:27.133134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b4d54-95b6-4cd0-bd38-12a2f72400da None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.133415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b4d54-95b6-4cd0-bd38-12a2f72400da None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.133686 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b4d54-95b6-4cd0-bd38-12a2f72400da None admin] PATH_INFO: `/v3/users/002fd00099394087b24dc6d26ef9fa9a` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.148629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b4d54-95b6-4cd0-bd38-12a2f72400da None admin] RBAC: Authorizing `identity:get_user(user_id=002fd00099394087b24dc6d26ef9fa9a)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:27.150671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b4d54-95b6-4cd0-bd38-12a2f72400da None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:27.152770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 120/243] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:26 2022] GET /identity/v3/users/002fd00099394087b24dc6d26ef9fa9a => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:27.158456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0551d55-1707-460f-bcd9-30ba81fb96b4 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:27.163330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:27.267774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0551d55-1707-460f-bcd9-30ba81fb96b4 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:27.310385 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255266ab-a1f4-48c5-af73-6818a03a85a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:27.311156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255266ab-a1f4-48c5-af73-6818a03a85a4 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.311436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255266ab-a1f4-48c5-af73-6818a03a85a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.311705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255266ab-a1f4-48c5-af73-6818a03a85a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.331370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4435a-ea7f-4516-8e2b-46a3fe9927a2 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.331893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4435a-ea7f-4516-8e2b-46a3fe9927a2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.331893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4435a-ea7f-4516-8e2b-46a3fe9927a2 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.333592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 121/244] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:27.345812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399938f-2d0d-4a50-b99c-402450c37765 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.346082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399938f-2d0d-4a50-b99c-402450c37765 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.346353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399938f-2d0d-4a50-b99c-402450c37765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.369723 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-8399938f-2d0d-4a50-b99c-402450c37765 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:27.371316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8399938f-2d0d-4a50-b99c-402450c37765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:27.406501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255266ab-a1f4-48c5-af73-6818a03a85a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:27.408015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255266ab-a1f4-48c5-af73-6818a03a85a4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:27.412274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:27.478462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 122/245] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:27.503184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399938f-2d0d-4a50-b99c-402450c37765 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:27.513978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:27.530355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 124/246] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:27.531657 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b8a6f-e0e8-46a6-8077-d10d2d049c61 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.531939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b8a6f-e0e8-46a6-8077-d10d2d049c61 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.532209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b8a6f-e0e8-46a6-8077-d10d2d049c61 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.538772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 125/247] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:27.540253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b8a6f-e0e8-46a6-8077-d10d2d049c61 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:27.626091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399938f-2d0d-4a50-b99c-402450c37765 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:27.662431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cea1b-ceca-47dd-a8cb-c93ab6d4733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:27.663086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cea1b-ceca-47dd-a8cb-c93ab6d4733b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.663363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cea1b-ceca-47dd-a8cb-c93ab6d4733b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.663630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cea1b-ceca-47dd-a8cb-c93ab6d4733b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.664767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b8a6f-e0e8-46a6-8077-d10d2d049c61 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:27.675671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cea1b-ceca-47dd-a8cb-c93ab6d4733b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-1c5cea1b-ceca-47dd-a8cb-c93ab6d4733b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:27.681959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:27.682825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:27.683639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 123/248] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:50:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:27.684260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:27.684791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca7289-559a-4514-939d-af1b51ba88ae None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.684966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca7289-559a-4514-939d-af1b51ba88ae None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.685316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca7289-559a-4514-939d-af1b51ba88ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.706113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0eca7289-559a-4514-939d-af1b51ba88ae None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:27.707428 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0eca7289-559a-4514-939d-af1b51ba88ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:27.713534 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c82bb-4129-4b3c-8c74-845199c9b92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:27.714170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c82bb-4129-4b3c-8c74-845199c9b92c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.714444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c82bb-4129-4b3c-8c74-845199c9b92c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.714713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c82bb-4129-4b3c-8c74-845199c9b92c None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.716044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c82bb-4129-4b3c-8c74-845199c9b92c None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:27.717374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c82bb-4129-4b3c-8c74-845199c9b92c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:27.719175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 126/249] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:27 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:27.720545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f495-fc53-40a2-bdf8-daa12fda1604 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.720824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f495-fc53-40a2-bdf8-daa12fda1604 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.721093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f495-fc53-40a2-bdf8-daa12fda1604 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.722278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 127/250] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:27.727111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78f495-fc53-40a2-bdf8-daa12fda1604 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:27.776069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:27.785876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 124/251] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:27.787349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca7289-559a-4514-939d-af1b51ba88ae None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:27.850537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78f495-fc53-40a2-bdf8-daa12fda1604 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:27.887338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb995c-37e8-4756-bd09-5212ebf76790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:27.888009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb995c-37e8-4756-bd09-5212ebf76790 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.888359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb995c-37e8-4756-bd09-5212ebf76790 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.888696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb995c-37e8-4756-bd09-5212ebf76790 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.889428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb995c-37e8-4756-bd09-5212ebf76790 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:27.889838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb995c-37e8-4756-bd09-5212ebf76790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:27.891307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb995c-37e8-4756-bd09-5212ebf76790 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:27.896056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca7289-559a-4514-939d-af1b51ba88ae None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:27.902182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 128/252] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:27.903501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d853c-fea7-4973-93a3-3fe13caeeb61 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.903784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d853c-fea7-4973-93a3-3fe13caeeb61 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.904058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d853c-fea7-4973-93a3-3fe13caeeb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.923849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-602d853c-fea7-4973-93a3-3fe13caeeb61 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:27.925028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-602d853c-fea7-4973-93a3-3fe13caeeb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:27.948095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f4002-e8f6-47cc-9bd4-1f22703f5a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:27.948974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f4002-e8f6-47cc-9bd4-1f22703f5a01 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:27.949434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f4002-e8f6-47cc-9bd4-1f22703f5a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:27.949771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f4002-e8f6-47cc-9bd4-1f22703f5a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:27.950571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f4002-e8f6-47cc-9bd4-1f22703f5a01 None admin] RBAC: Adding query filter params (role.id=053f81c391444db1b6409c4eb953dc35, scope.project.id=d254037584f44b54b09c2b3e76807918, user.id=002fd00099394087b24dc6d26ef9fa9a) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:27.950947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f4002-e8f6-47cc-9bd4-1f22703f5a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:27.952558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f4002-e8f6-47cc-9bd4-1f22703f5a01 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:27.959038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 125/253] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:27 2022] GET /identity/v3/role_assignments?user.id=002fd00099394087b24dc6d26ef9fa9a&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=053f81c391444db1b6409c4eb953dc35 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:27.966263 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:27.970999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f4002-e8f6-47cc-9bd4-1f22703f5a01 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:28.036622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:28.049557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 129/254] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:28.050816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d853c-fea7-4973-93a3-3fe13caeeb61 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:28.090632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f4002-e8f6-47cc-9bd4-1f22703f5a01 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:28.128719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5e2f3-38b1-417f-8d63-f4864ff35f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:28.129473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5e2f3-38b1-417f-8d63-f4864ff35f0a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:28.129818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5e2f3-38b1-417f-8d63-f4864ff35f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:28.130160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5e2f3-38b1-417f-8d63-f4864ff35f0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:28.142142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5e2f3-38b1-417f-8d63-f4864ff35f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-2da5e2f3-38b1-417f-8d63-f4864ff35f0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:28.147630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:28.148522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:28.150149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 126/255] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:28.151492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5e2f3-38b1-417f-8d63-f4864ff35f0a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:28.260838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5e2f3-38b1-417f-8d63-f4864ff35f0a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:28.268779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d853c-fea7-4973-93a3-3fe13caeeb61 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:28.303341 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:28.316031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69841657-2d69-4d49-a02a-b2362fbeb756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:28.316716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69841657-2d69-4d49-a02a-b2362fbeb756 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:28.316978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69841657-2d69-4d49-a02a-b2362fbeb756 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:28.317264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69841657-2d69-4d49-a02a-b2362fbeb756 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:28.328173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b4bdf-5aff-4c62-8660-79c5df6da143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:28.328823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b4bdf-5aff-4c62-8660-79c5df6da143 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:28.329105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b4bdf-5aff-4c62-8660-79c5df6da143 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:28.329379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b4bdf-5aff-4c62-8660-79c5df6da143 None admin] PATH_INFO: `/v3/users/b118ad55fa4941a39e84abb92c57461a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:28.331174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b4bdf-5aff-4c62-8660-79c5df6da143 None admin] RBAC: Authorizing `identity:get_user(user_id=b118ad55fa4941a39e84abb92c57461a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:28.332615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b4bdf-5aff-4c62-8660-79c5df6da143 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:28.338186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 130/256] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:28 2022] GET /identity/v3/users/b118ad55fa4941a39e84abb92c57461a => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:28.338988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b4bdf-5aff-4c62-8660-79c5df6da143 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:28.341763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69841657-2d69-4d49-a02a-b2362fbeb756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-69841657-2d69-4d49-a02a-b2362fbeb756 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:28.349018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:28.349883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:28.350646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:28.357693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 127/257] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:28.358592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69841657-2d69-4d49-a02a-b2362fbeb756 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:28.412790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:28.548567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b4bdf-5aff-4c62-8660-79c5df6da143 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:28.565404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69841657-2d69-4d49-a02a-b2362fbeb756 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:28.603414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7d7f9-313f-44a2-a1e6-d4f1faf59c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:28.604066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7d7f9-313f-44a2-a1e6-d4f1faf59c74 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:28.604349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7d7f9-313f-44a2-a1e6-d4f1faf59c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:28.604622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7d7f9-313f-44a2-a1e6-d4f1faf59c74 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:28.605518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7d7f9-313f-44a2-a1e6-d4f1faf59c74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:28.605859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7d7f9-313f-44a2-a1e6-d4f1faf59c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:28.607445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7d7f9-313f-44a2-a1e6-d4f1faf59c74 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:28.617185 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 131/258] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:28.619221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7d7f9-313f-44a2-a1e6-d4f1faf59c74 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:28.648446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac301b4-3160-4601-9b26-27a1cc300e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:28.649561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac301b4-3160-4601-9b26-27a1cc300e8a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:28.650049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac301b4-3160-4601-9b26-27a1cc300e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:28.650568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac301b4-3160-4601-9b26-27a1cc300e8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:28.755187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:28.756161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:28.760810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7d7f9-313f-44a2-a1e6-d4f1faf59c74 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:28.796074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac301b4-3160-4601-9b26-27a1cc300e8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:28.797555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac301b4-3160-4601-9b26-27a1cc300e8a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:28.830763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cada3-c73d-4321-9d7a-bc9113a1b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:28.831420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cada3-c73d-4321-9d7a-bc9113a1b96b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:28.831701 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cada3-c73d-4321-9d7a-bc9113a1b96b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:28.831975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cada3-c73d-4321-9d7a-bc9113a1b96b None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:28.832640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cada3-c73d-4321-9d7a-bc9113a1b96b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:28.832941 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cada3-c73d-4321-9d7a-bc9113a1b96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:28.835330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cada3-c73d-4321-9d7a-bc9113a1b96b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:28.851046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 132/259] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:28.851977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cada3-c73d-4321-9d7a-bc9113a1b96b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:29.001779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 128/260] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:29.002707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac301b4-3160-4601-9b26-27a1cc300e8a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:29.042690 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cada3-c73d-4321-9d7a-bc9113a1b96b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:29.108335 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81470f-4c51-4a45-9727-9724b758f9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:29.109028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81470f-4c51-4a45-9727-9724b758f9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.109310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81470f-4c51-4a45-9727-9724b758f9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.109582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81470f-4c51-4a45-9727-9724b758f9d1 None admin] PATH_INFO: `/v3/users/0da08bc23fe548e4a41f4979cc6c525c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.111015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81470f-4c51-4a45-9727-9724b758f9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=0da08bc23fe548e4a41f4979cc6c525c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:29.113320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81470f-4c51-4a45-9727-9724b758f9d1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:29.115105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 133/261] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:28 2022] GET /identity/v3/users/0da08bc23fe548e4a41f4979cc6c525c => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:29.116542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81470f-4c51-4a45-9727-9724b758f9d1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:29.159295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:29.170679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:29.194632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac301b4-3160-4601-9b26-27a1cc300e8a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:29.255068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81470f-4c51-4a45-9727-9724b758f9d1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:29.264638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efbe30-e884-4a03-a68f-2d8d8db5489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:29.265291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efbe30-e884-4a03-a68f-2d8d8db5489d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.265568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efbe30-e884-4a03-a68f-2d8d8db5489d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.265844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efbe30-e884-4a03-a68f-2d8d8db5489d None admin] PATH_INFO: `/v3/users/b118ad55fa4941a39e84abb92c57461a` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.267003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efbe30-e884-4a03-a68f-2d8d8db5489d None admin] RBAC: Authorizing `identity:get_user(user_id=b118ad55fa4941a39e84abb92c57461a)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:29.268478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efbe30-e884-4a03-a68f-2d8d8db5489d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:29.270067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 129/262] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:29 2022] GET /identity/v3/users/b118ad55fa4941a39e84abb92c57461a => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:29.270830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efbe30-e884-4a03-a68f-2d8d8db5489d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:29.300782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43621ef6-d223-4f2d-bb3a-e92a2a312dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:29.301397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43621ef6-d223-4f2d-bb3a-e92a2a312dce None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.301624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43621ef6-d223-4f2d-bb3a-e92a2a312dce None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.301896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43621ef6-d223-4f2d-bb3a-e92a2a312dce None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.307332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43621ef6-d223-4f2d-bb3a-e92a2a312dce None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:29.308781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43621ef6-d223-4f2d-bb3a-e92a2a312dce None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:29.314224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 134/263] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:29 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:29.315116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43621ef6-d223-4f2d-bb3a-e92a2a312dce None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:29.439293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:29.450117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43621ef6-d223-4f2d-bb3a-e92a2a312dce None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:29.486782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9b1be-3119-4f0b-a190-7845263cc0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:29.487802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b1be-3119-4f0b-a190-7845263cc0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.488276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b1be-3119-4f0b-a190-7845263cc0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.488746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b1be-3119-4f0b-a190-7845263cc0c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.543202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efbe30-e884-4a03-a68f-2d8d8db5489d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:29.561904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9b1be-3119-4f0b-a190-7845263cc0c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:29.563701 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9b1be-3119-4f0b-a190-7845263cc0c5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:29.602971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd260f0-e434-4d57-ba29-4aca3045dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:29.603663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd260f0-e434-4d57-ba29-4aca3045dc70 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.603949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd260f0-e434-4d57-ba29-4aca3045dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.604219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd260f0-e434-4d57-ba29-4aca3045dc70 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.605360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd260f0-e434-4d57-ba29-4aca3045dc70 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:29.609944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd260f0-e434-4d57-ba29-4aca3045dc70 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:29.611683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 130/264] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:29 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:29.612443 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd260f0-e434-4d57-ba29-4aca3045dc70 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:29.634788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:29.665589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:29.677570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 135/265] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:29.678807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48006-9b3b-4dd4-9b33-8627f890428d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.679121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48006-9b3b-4dd4-9b33-8627f890428d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.679395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48006-9b3b-4dd4-9b33-8627f890428d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.680660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 136/266] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:29.681333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48006-9b3b-4dd4-9b33-8627f890428d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:29.721354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd260f0-e434-4d57-ba29-4aca3045dc70 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:29.755806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868d57d-e02d-449b-98e8-e3fb97fd5f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:29.756395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868d57d-e02d-449b-98e8-e3fb97fd5f41 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.756665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868d57d-e02d-449b-98e8-e3fb97fd5f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.756928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868d57d-e02d-449b-98e8-e3fb97fd5f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.757641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868d57d-e02d-449b-98e8-e3fb97fd5f41 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=d254037584f44b54b09c2b3e76807918, user.id=b118ad55fa4941a39e84abb92c57461a) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:29.757942 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868d57d-e02d-449b-98e8-e3fb97fd5f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:29.759905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868d57d-e02d-449b-98e8-e3fb97fd5f41 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:29.775102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 131/267] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:29 2022] GET /identity/v3/role_assignments?user.id=b118ad55fa4941a39e84abb92c57461a&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=8417343e9ef64991bcf51bb9692da102 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:29.775102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025bf69-14c9-4b62-a964-16ba4001b20f None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.775102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025bf69-14c9-4b62-a964-16ba4001b20f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.775102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025bf69-14c9-4b62-a964-16ba4001b20f None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.775102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 132/268] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:29.775102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025bf69-14c9-4b62-a964-16ba4001b20f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:29.798011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48006-9b3b-4dd4-9b33-8627f890428d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:29.845602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7baa7f-e2da-4a6d-a0c5-a09d5c783564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:29.846300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7baa7f-e2da-4a6d-a0c5-a09d5c783564 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.846580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7baa7f-e2da-4a6d-a0c5-a09d5c783564 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.846849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7baa7f-e2da-4a6d-a0c5-a09d5c783564 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918/users/0da08bc23fe548e4a41f4979cc6c525c/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.848934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7baa7f-e2da-4a6d-a0c5-a09d5c783564 None admin] RBAC: Authorizing `identity:create_grant(project_id=d254037584f44b54b09c2b3e76807918, user_id=0da08bc23fe548e4a41f4979cc6c525c, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:29.850456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7baa7f-e2da-4a6d-a0c5-a09d5c783564 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:29.875065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 137/269] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:50:29 2022] PUT /identity/v3/projects/d254037584f44b54b09c2b3e76807918/users/0da08bc23fe548e4a41f4979cc6c525c/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:29.876710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443413-4509-4156-abdd-807f7a5d69a2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:29.877001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443413-4509-4156-abdd-807f7a5d69a2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:29.877901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443413-4509-4156-abdd-807f7a5d69a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:29.882771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:29.917310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-49443413-4509-4156-abdd-807f7a5d69a2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:29.918893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-49443413-4509-4156-abdd-807f7a5d69a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:29.929102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025bf69-14c9-4b62-a964-16ba4001b20f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:30.005668 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1e5d0-a169-4122-aec6-b01d46e18b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:30.006323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1e5d0-a169-4122-aec6-b01d46e18b67 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.006605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1e5d0-a169-4122-aec6-b01d46e18b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.006880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1e5d0-a169-4122-aec6-b01d46e18b67 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.007953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:30.008702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1e5d0-a169-4122-aec6-b01d46e18b67 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:30.010187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1e5d0-a169-4122-aec6-b01d46e18b67 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:30.012097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 133/270] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:29 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:30.013122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b9e0a-2739-4084-b78e-05212b63f8fe None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.013428 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b9e0a-2739-4084-b78e-05212b63f8fe None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.013951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b9e0a-2739-4084-b78e-05212b63f8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.037539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-cb2b9e0a-2739-4084-b78e-05212b63f8fe None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:30.038822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-cb2b9e0a-2739-4084-b78e-05212b63f8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:30.039665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 138/271] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:30.040473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49443413-4509-4156-abdd-807f7a5d69a2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:30.123627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 134/272] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:30.124788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b9e0a-2739-4084-b78e-05212b63f8fe None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:30.176178 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49443413-4509-4156-abdd-807f7a5d69a2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:30.214718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ac824-43c9-4592-b749-5dd5162fac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:30.215817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ac824-43c9-4592-b749-5dd5162fac89 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.216288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ac824-43c9-4592-b749-5dd5162fac89 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.216769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ac824-43c9-4592-b749-5dd5162fac89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.217680 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ac824-43c9-4592-b749-5dd5162fac89 None admin] RBAC: Adding query filter params (role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=d254037584f44b54b09c2b3e76807918, user.id=b118ad55fa4941a39e84abb92c57461a) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:30.218247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ac824-43c9-4592-b749-5dd5162fac89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:30.220014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ac824-43c9-4592-b749-5dd5162fac89 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:30.226619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 139/273] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:30 2022] GET /identity/v3/role_assignments?user.id=b118ad55fa4941a39e84abb92c57461a&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:30.227752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ac824-43c9-4592-b749-5dd5162fac89 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:30.228352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:30.254330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b9e0a-2739-4084-b78e-05212b63f8fe None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:30.295839 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f031fd7-e4eb-48a0-9bfc-8051cc2dd65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:30.296487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f031fd7-e4eb-48a0-9bfc-8051cc2dd65b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.296765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f031fd7-e4eb-48a0-9bfc-8051cc2dd65b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.297034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f031fd7-e4eb-48a0-9bfc-8051cc2dd65b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.298355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f031fd7-e4eb-48a0-9bfc-8051cc2dd65b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:30.300315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f031fd7-e4eb-48a0-9bfc-8051cc2dd65b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:30.302439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 135/274] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:30.312038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f031fd7-e4eb-48a0-9bfc-8051cc2dd65b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:30.325788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:30.391995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ac824-43c9-4592-b749-5dd5162fac89 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:30.432019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b735b71-eb03-4d3f-9c73-2b52b8c8d707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:30.433013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b735b71-eb03-4d3f-9c73-2b52b8c8d707 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.433628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b735b71-eb03-4d3f-9c73-2b52b8c8d707 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.434085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b735b71-eb03-4d3f-9c73-2b52b8c8d707 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.447070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b735b71-eb03-4d3f-9c73-2b52b8c8d707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9b735b71-eb03-4d3f-9c73-2b52b8c8d707 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:30.452574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:30.453510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:30.454281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 140/275] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:30.455070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c61ab-0fb6-4e5f-a4af-b53f269aabe4 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.455710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c61ab-0fb6-4e5f-a4af-b53f269aabe4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.455710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c61ab-0fb6-4e5f-a4af-b53f269aabe4 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.456854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 141/276] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:30.460453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c61ab-0fb6-4e5f-a4af-b53f269aabe4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:30.486485 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f031fd7-e4eb-48a0-9bfc-8051cc2dd65b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:30.529579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d2baa-09af-47c5-95d9-6dfc3915da94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:30.530197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d2baa-09af-47c5-95d9-6dfc3915da94 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.530473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d2baa-09af-47c5-95d9-6dfc3915da94 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.530910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d2baa-09af-47c5-95d9-6dfc3915da94 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.531918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d2baa-09af-47c5-95d9-6dfc3915da94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:30.533367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d2baa-09af-47c5-95d9-6dfc3915da94 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:30.561551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 136/277] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:30 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:50:30.563639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc5592-6e7e-4d63-9118-bbb14bf8cbd2 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.564148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc5592-6e7e-4d63-9118-bbb14bf8cbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.564697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc5592-6e7e-4d63-9118-bbb14bf8cbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.572571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:30.590976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-81dc5592-6e7e-4d63-9118-bbb14bf8cbd2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:30.592283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-81dc5592-6e7e-4d63-9118-bbb14bf8cbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:30.607440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c61ab-0fb6-4e5f-a4af-b53f269aabe4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:30.616687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:30.650284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0011a3c-9845-433e-8de6-b11d8b625957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:30.650924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0011a3c-9845-433e-8de6-b11d8b625957 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.652296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0011a3c-9845-433e-8de6-b11d8b625957 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.652651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0011a3c-9845-433e-8de6-b11d8b625957 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.653487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0011a3c-9845-433e-8de6-b11d8b625957 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:30.653884 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0011a3c-9845-433e-8de6-b11d8b625957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:30.655866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0011a3c-9845-433e-8de6-b11d8b625957 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:30.666245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 142/278] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:30.672707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0011a3c-9845-433e-8de6-b11d8b625957 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:30.681508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 137/279] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:30.705858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc5592-6e7e-4d63-9118-bbb14bf8cbd2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:30.800238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0011a3c-9845-433e-8de6-b11d8b625957 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:30.829237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc5592-6e7e-4d63-9118-bbb14bf8cbd2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:30.841196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a2a6c-94bd-4fd8-91ff-f1b11fe90424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:30.841196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a2a6c-94bd-4fd8-91ff-f1b11fe90424 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.841196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a2a6c-94bd-4fd8-91ff-f1b11fe90424 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.841196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a2a6c-94bd-4fd8-91ff-f1b11fe90424 None admin] PATH_INFO: `/v3/users/b9925d98e992405e8e8dd35edbc3de67` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.860895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a2a6c-94bd-4fd8-91ff-f1b11fe90424 None admin] RBAC: Authorizing `identity:get_user(user_id=b9925d98e992405e8e8dd35edbc3de67)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:30.862353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a2a6c-94bd-4fd8-91ff-f1b11fe90424 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:30.864243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 143/280] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:30 2022] GET /identity/v3/users/b9925d98e992405e8e8dd35edbc3de67 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:30.871836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:30.872370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03041483-908e-4dfb-b883-fd08d375743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:30.873047 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a2a6c-94bd-4fd8-91ff-f1b11fe90424 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:30.873258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03041483-908e-4dfb-b883-fd08d375743a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:30.873509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03041483-908e-4dfb-b883-fd08d375743a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:30.873758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03041483-908e-4dfb-b883-fd08d375743a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:30.887741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03041483-908e-4dfb-b883-fd08d375743a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-03041483-908e-4dfb-b883-fd08d375743a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:30.893544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:30.894429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:30.895224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:30.897613 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 138/281] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:30.901074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03041483-908e-4dfb-b883-fd08d375743a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:30.908073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:30.994998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a2a6c-94bd-4fd8-91ff-f1b11fe90424 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:31.033157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb7e98-380f-480a-a02d-63d4e71b83ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:31.034260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb7e98-380f-480a-a02d-63d4e71b83ce None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:31.034771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb7e98-380f-480a-a02d-63d4e71b83ce None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:31.035309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb7e98-380f-480a-a02d-63d4e71b83ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:31.036373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb7e98-380f-480a-a02d-63d4e71b83ce None admin] RBAC: Adding query filter params (role.id=7694dcfc7d6547abb6565cb9e726eacd, user.id=b9925d98e992405e8e8dd35edbc3de67, scope.system=all) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:31.036997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb7e98-380f-480a-a02d-63d4e71b83ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:31.038698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb7e98-380f-480a-a02d-63d4e71b83ce None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:31.040474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03041483-908e-4dfb-b883-fd08d375743a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:31.046880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 144/282] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 10 09:50:30 2022] GET /identity/v3/role_assignments?user.id=b9925d98e992405e8e8dd35edbc3de67&scope.system=all&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:31.088565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942382e7-1821-4c71-9c48-6c3b7ccf6834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:31.089189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942382e7-1821-4c71-9c48-6c3b7ccf6834 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:31.089465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942382e7-1821-4c71-9c48-6c3b7ccf6834 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:31.089737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942382e7-1821-4c71-9c48-6c3b7ccf6834 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:31.090416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942382e7-1821-4c71-9c48-6c3b7ccf6834 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:31.090717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942382e7-1821-4c71-9c48-6c3b7ccf6834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:31.092212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942382e7-1821-4c71-9c48-6c3b7ccf6834 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:31.102419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 139/283] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:31.108272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb7e98-380f-480a-a02d-63d4e71b83ce None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:31.217304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:31.225124 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb7e98-380f-480a-a02d-63d4e71b83ce None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:31.260645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc56b61-7803-4b0a-af90-37ecb767c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:31.261284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc56b61-7803-4b0a-af90-37ecb767c253 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:31.261568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc56b61-7803-4b0a-af90-37ecb767c253 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:31.261889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc56b61-7803-4b0a-af90-37ecb767c253 None admin] PATH_INFO: `/v3/users/002fd00099394087b24dc6d26ef9fa9a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:31.263099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc56b61-7803-4b0a-af90-37ecb767c253 None admin] RBAC: Authorizing `identity:get_user(user_id=002fd00099394087b24dc6d26ef9fa9a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:31.264579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc56b61-7803-4b0a-af90-37ecb767c253 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:31.266175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 145/284] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:31 2022] GET /identity/v3/users/002fd00099394087b24dc6d26ef9fa9a => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:31.271684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942382e7-1821-4c71-9c48-6c3b7ccf6834 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:31.333969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:31.385882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942382e7-1821-4c71-9c48-6c3b7ccf6834 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:31.446213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ec44c-57c0-4c7a-a64f-3e00cbdd0b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:31.446902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ec44c-57c0-4c7a-a64f-3e00cbdd0b45 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:31.447207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ec44c-57c0-4c7a-a64f-3e00cbdd0b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:31.447478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ec44c-57c0-4c7a-a64f-3e00cbdd0b45 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:31.448631 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ec44c-57c0-4c7a-a64f-3e00cbdd0b45 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:31.450083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ec44c-57c0-4c7a-a64f-3e00cbdd0b45 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:31.452069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 140/285] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:31 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:31.457887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc56b61-7803-4b0a-af90-37ecb767c253 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:31.578347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc56b61-7803-4b0a-af90-37ecb767c253 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:31.626351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03ccf0-0840-470e-824c-1ac6483e221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:31.627282 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03ccf0-0840-470e-824c-1ac6483e221c None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:31.627733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03ccf0-0840-470e-824c-1ac6483e221c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:31.628147 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03ccf0-0840-470e-824c-1ac6483e221c None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918/users/002fd00099394087b24dc6d26ef9fa9a/roles/053f81c391444db1b6409c4eb953dc35` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:31.630226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03ccf0-0840-470e-824c-1ac6483e221c None admin] RBAC: Authorizing `identity:create_grant(project_id=d254037584f44b54b09c2b3e76807918, user_id=002fd00099394087b24dc6d26ef9fa9a, role_id=053f81c391444db1b6409c4eb953dc35)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:31.631839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03ccf0-0840-470e-824c-1ac6483e221c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:31.648738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 146/286] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:50:31 2022] PUT /identity/v3/projects/d254037584f44b54b09c2b3e76807918/users/002fd00099394087b24dc6d26ef9fa9a/roles/053f81c391444db1b6409c4eb953dc35 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:32.437378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b34dac-6b23-4a7c-9456-ec8dde1a2a8c None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:32.437876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b34dac-6b23-4a7c-9456-ec8dde1a2a8c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:32.438182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b34dac-6b23-4a7c-9456-ec8dde1a2a8c None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:32.440580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 141/287] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:32.445492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609582b4-6ccd-4f9a-b5f4-a1f97460bc59 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:32.446022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609582b4-6ccd-4f9a-b5f4-a1f97460bc59 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:32.446494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609582b4-6ccd-4f9a-b5f4-a1f97460bc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:32.475042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-609582b4-6ccd-4f9a-b5f4-a1f97460bc59 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:32.477558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-609582b4-6ccd-4f9a-b5f4-a1f97460bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:32.491541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:32.547736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 147/288] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:32.581757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b34dac-6b23-4a7c-9456-ec8dde1a2a8c None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:32.713360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b34dac-6b23-4a7c-9456-ec8dde1a2a8c None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:32.740795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:32.755415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c5887-7041-4e04-9988-03e717ab9027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:32.756185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c5887-7041-4e04-9988-03e717ab9027 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:32.756530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c5887-7041-4e04-9988-03e717ab9027 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:32.757014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c5887-7041-4e04-9988-03e717ab9027 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:32.771479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c5887-7041-4e04-9988-03e717ab9027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-dd4c5887-7041-4e04-9988-03e717ab9027 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:32.776757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:32.777615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:32.778393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 142/289] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:32.787897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609582b4-6ccd-4f9a-b5f4-a1f97460bc59 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:32.929730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609582b4-6ccd-4f9a-b5f4-a1f97460bc59 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:32.974627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b83ab-3759-47ca-8268-f7615ccfc4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:32.975183 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b83ab-3759-47ca-8268-f7615ccfc4ba None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:32.975465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b83ab-3759-47ca-8268-f7615ccfc4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:32.975740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b83ab-3759-47ca-8268-f7615ccfc4ba None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:32.976416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b83ab-3759-47ca-8268-f7615ccfc4ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:32.976746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b83ab-3759-47ca-8268-f7615ccfc4ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:32.978136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b83ab-3759-47ca-8268-f7615ccfc4ba None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:32.990553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 148/290] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:33.000107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c5887-7041-4e04-9988-03e717ab9027 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:33.141369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c5887-7041-4e04-9988-03e717ab9027 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:33.186228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf26454-f60b-4382-808a-d104523eefb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:33.186860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf26454-f60b-4382-808a-d104523eefb2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.187158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf26454-f60b-4382-808a-d104523eefb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.187427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf26454-f60b-4382-808a-d104523eefb2 None admin] PATH_INFO: `/v3/users/0da08bc23fe548e4a41f4979cc6c525c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.188540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf26454-f60b-4382-808a-d104523eefb2 None admin] RBAC: Authorizing `identity:get_user(user_id=0da08bc23fe548e4a41f4979cc6c525c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:33.189983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf26454-f60b-4382-808a-d104523eefb2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:33.191859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 143/291] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:32 2022] GET /identity/v3/users/0da08bc23fe548e4a41f4979cc6c525c => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:33.198037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b83ab-3759-47ca-8268-f7615ccfc4ba None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:33.226153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:33.238081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b5d9e-95d5-4c75-9afc-5a5ae96a22aa None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.238331 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b5d9e-95d5-4c75-9afc-5a5ae96a22aa None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.238634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b5d9e-95d5-4c75-9afc-5a5ae96a22aa None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.239991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 144/292] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:33.246660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3265e3e-41c6-4525-a125-6e953d5a4be6 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.246944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3265e3e-41c6-4525-a125-6e953d5a4be6 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.247257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3265e3e-41c6-4525-a125-6e953d5a4be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.266776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f3265e3e-41c6-4525-a125-6e953d5a4be6 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:33.267983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f3265e3e-41c6-4525-a125-6e953d5a4be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:33.316874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:33.341352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 145/293] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:33.342136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b83ab-3759-47ca-8268-f7615ccfc4ba None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:33.364043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3265e3e-41c6-4525-a125-6e953d5a4be6 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:33.382356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7773c-a2a2-4433-a817-77ba1acb1fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:33.383582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7773c-a2a2-4433-a817-77ba1acb1fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.383875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7773c-a2a2-4433-a817-77ba1acb1fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.384145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7773c-a2a2-4433-a817-77ba1acb1fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.484443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7773c-a2a2-4433-a817-77ba1acb1fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:33.485863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7773c-a2a2-4433-a817-77ba1acb1fd9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:33.499508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:33.517316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3265e3e-41c6-4525-a125-6e953d5a4be6 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:33.552498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b9652-9ea6-4581-9a61-a4d11401e1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:33.553127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b9652-9ea6-4581-9a61-a4d11401e1cc None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.553462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b9652-9ea6-4581-9a61-a4d11401e1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.553736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b9652-9ea6-4581-9a61-a4d11401e1cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.554785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b9652-9ea6-4581-9a61-a4d11401e1cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:33.556876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b9652-9ea6-4581-9a61-a4d11401e1cc None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:33.558623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 146/294] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:33.560465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2e0b9-c36b-4862-a2de-1d5a78c1c440 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.561548 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2e0b9-c36b-4862-a2de-1d5a78c1c440 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.562233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2e0b9-c36b-4862-a2de-1d5a78c1c440 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.564030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 147/295] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:33.566078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3267a-23ba-4e6f-b5ca-8890f3078222 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.566534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3267a-23ba-4e6f-b5ca-8890f3078222 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.566963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3267a-23ba-4e6f-b5ca-8890f3078222 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.568371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 148/296] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:33.569237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3267a-23ba-4e6f-b5ca-8890f3078222 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:33.596419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 149/297] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:33.597655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d16b6-cdf6-4dce-a018-6077fc63e39a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.597972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d16b6-cdf6-4dce-a018-6077fc63e39a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.598209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d16b6-cdf6-4dce-a018-6077fc63e39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.615891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7a7d16b6-cdf6-4dce-a018-6077fc63e39a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:33.617065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7a7d16b6-cdf6-4dce-a018-6077fc63e39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:33.623837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:33.682601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3267a-23ba-4e6f-b5ca-8890f3078222 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:33.694975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 150/298] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:33.694975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b5c97-365c-457a-99ae-8703c1ecff9a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.694975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b5c97-365c-457a-99ae-8703c1ecff9a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.694975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b5c97-365c-457a-99ae-8703c1ecff9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.713705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3d2b5c97-365c-457a-99ae-8703c1ecff9a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:33.714989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3d2b5c97-365c-457a-99ae-8703c1ecff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:33.717781 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f073c-2ae0-4902-8a86-7db83d531bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:33.718467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f073c-2ae0-4902-8a86-7db83d531bb9 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.718750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f073c-2ae0-4902-8a86-7db83d531bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.719050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f073c-2ae0-4902-8a86-7db83d531bb9 None admin] PATH_INFO: `/v3/groups` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.719673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f073c-2ae0-4902-8a86-7db83d531bb9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:33.721099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f073c-2ae0-4902-8a86-7db83d531bb9 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:33.738120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 149/299] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Aug 10 09:50:33 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:33.739677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f073c-2ae0-4902-8a86-7db83d531bb9 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:33.755347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:33.789779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 151/300] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:33.790758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b5c97-365c-457a-99ae-8703c1ecff9a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:33.899586 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f073c-2ae0-4902-8a86-7db83d531bb9 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:33.908317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b5c97-365c-457a-99ae-8703c1ecff9a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:33.914014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:33.946493 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ef994-700c-491c-a274-15691486f000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:33.947197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ef994-700c-491c-a274-15691486f000 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.947540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ef994-700c-491c-a274-15691486f000 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.947897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ef994-700c-491c-a274-15691486f000 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.955168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25cf3d-a3f3-4dd2-abf8-51a599fac724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:33.956091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25cf3d-a3f3-4dd2-abf8-51a599fac724 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:33.956542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25cf3d-a3f3-4dd2-abf8-51a599fac724 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:33.957239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25cf3d-a3f3-4dd2-abf8-51a599fac724 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:33.958930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25cf3d-a3f3-4dd2-abf8-51a599fac724 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:33.960718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25cf3d-a3f3-4dd2-abf8-51a599fac724 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:33.962404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ef994-700c-491c-a274-15691486f000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:33.963340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 150/301] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:33 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:33.964124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25cf3d-a3f3-4dd2-abf8-51a599fac724 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6f5ef994-700c-491c-a274-15691486f000 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:33.971181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:33.972094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:33.972864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:33.975588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 152/302] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:33.976408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ef994-700c-491c-a274-15691486f000 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:34.027481 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:34.090694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25cf3d-a3f3-4dd2-abf8-51a599fac724 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:34.121040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ef994-700c-491c-a274-15691486f000 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:34.131244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f3db-addd-4950-b0b4-1b4300848206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:34.132443 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f3db-addd-4950-b0b4-1b4300848206 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.133078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f3db-addd-4950-b0b4-1b4300848206 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.133540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f3db-addd-4950-b0b4-1b4300848206 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.146203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7f3db-addd-4950-b0b4-1b4300848206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-d9b7f3db-addd-4950-b0b4-1b4300848206 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:34.154128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:34.155022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:34.155799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 151/303] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:34.156958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f3db-addd-4950-b0b4-1b4300848206 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:34.158287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d946ca8-7b5c-4598-9a56-f530e30d2861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:34.158898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d946ca8-7b5c-4598-9a56-f530e30d2861 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.159261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d946ca8-7b5c-4598-9a56-f530e30d2861 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.159542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d946ca8-7b5c-4598-9a56-f530e30d2861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.160367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d946ca8-7b5c-4598-9a56-f530e30d2861 None admin] RBAC: Adding query filter params (role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=d254037584f44b54b09c2b3e76807918, user.id=0da08bc23fe548e4a41f4979cc6c525c) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:34.160683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d946ca8-7b5c-4598-9a56-f530e30d2861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:34.162199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d946ca8-7b5c-4598-9a56-f530e30d2861 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:34.169146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 153/304] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:33 2022] GET /identity/v3/role_assignments?user.id=0da08bc23fe548e4a41f4979cc6c525c&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:34.170100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d946ca8-7b5c-4598-9a56-f530e30d2861 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:34.197552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:34.272222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:34.273553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f3db-addd-4950-b0b4-1b4300848206 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:34.280463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d946ca8-7b5c-4598-9a56-f530e30d2861 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:34.313691 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306749b-4676-40c7-8aa4-3cc733d20a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:34.314413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306749b-4676-40c7-8aa4-3cc733d20a7f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.314778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306749b-4676-40c7-8aa4-3cc733d20a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.315082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306749b-4676-40c7-8aa4-3cc733d20a7f None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.315948 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306749b-4676-40c7-8aa4-3cc733d20a7f None admin] RBAC: Adding query filter params (name=member) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:34.316351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306749b-4676-40c7-8aa4-3cc733d20a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:34.317737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fbf53-ffb6-48a1-baaf-faa7aea4c799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:34.317947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306749b-4676-40c7-8aa4-3cc733d20a7f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:34.318495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fbf53-ffb6-48a1-baaf-faa7aea4c799 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.318787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fbf53-ffb6-48a1-baaf-faa7aea4c799 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.319087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fbf53-ffb6-48a1-baaf-faa7aea4c799 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.319763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fbf53-ffb6-48a1-baaf-faa7aea4c799 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:34.320066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fbf53-ffb6-48a1-baaf-faa7aea4c799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:34.323740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fbf53-ffb6-48a1-baaf-faa7aea4c799 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:34.328233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 152/305] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:34.333992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 154/306] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:34.334245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306749b-4676-40c7-8aa4-3cc733d20a7f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:34.344171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fbf53-ffb6-48a1-baaf-faa7aea4c799 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:34.459856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:34.463967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306749b-4676-40c7-8aa4-3cc733d20a7f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:34.506630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fbf53-ffb6-48a1-baaf-faa7aea4c799 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:34.509534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69710b8f-421a-4662-87f6-27a101ac0d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:34.510175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710b8f-421a-4662-87f6-27a101ac0d3e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.510451 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710b8f-421a-4662-87f6-27a101ac0d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.510721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710b8f-421a-4662-87f6-27a101ac0d3e None admin] PATH_INFO: `/v3/users/b9925d98e992405e8e8dd35edbc3de67` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.511889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69710b8f-421a-4662-87f6-27a101ac0d3e None admin] RBAC: Authorizing `identity:get_user(user_id=b9925d98e992405e8e8dd35edbc3de67)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:34.513339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69710b8f-421a-4662-87f6-27a101ac0d3e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:34.515324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 153/307] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:34 2022] GET /identity/v3/users/b9925d98e992405e8e8dd35edbc3de67 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:34.521006 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69710b8f-421a-4662-87f6-27a101ac0d3e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:34.550529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ba9ef-3a2f-4dd5-b6c0-c22d3d8d64da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:34.551803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ba9ef-3a2f-4dd5-b6c0-c22d3d8d64da None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.552668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ba9ef-3a2f-4dd5-b6c0-c22d3d8d64da None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.553140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ba9ef-3a2f-4dd5-b6c0-c22d3d8d64da None admin] PATH_INFO: `/v3/users/04ab4ca23e7a4bfdb4bb717732648e44` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.572336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ba9ef-3a2f-4dd5-b6c0-c22d3d8d64da None admin] RBAC: Authorizing `identity:get_user(user_id=04ab4ca23e7a4bfdb4bb717732648e44)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:34.574858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ba9ef-3a2f-4dd5-b6c0-c22d3d8d64da None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:34.576863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 155/308] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:34 2022] GET /identity/v3/users/04ab4ca23e7a4bfdb4bb717732648e44 => generated 293 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:34.578189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13d7d9-3060-472b-be80-c3d1ff1a1801 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.578467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13d7d9-3060-472b-be80-c3d1ff1a1801 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.578737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13d7d9-3060-472b-be80-c3d1ff1a1801 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.580084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 156/309] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:34.584628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13d7d9-3060-472b-be80-c3d1ff1a1801 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:34.602423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:34.643646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69710b8f-421a-4662-87f6-27a101ac0d3e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:34.683108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ee47c-df91-42c1-9d66-47a5ddc3eeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:34.684207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ee47c-df91-42c1-9d66-47a5ddc3eeb1 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.684510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ee47c-df91-42c1-9d66-47a5ddc3eeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.684925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ee47c-df91-42c1-9d66-47a5ddc3eeb1 None admin] PATH_INFO: `/v3/system/users/b9925d98e992405e8e8dd35edbc3de67/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.686553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ee47c-df91-42c1-9d66-47a5ddc3eeb1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b9925d98e992405e8e8dd35edbc3de67, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:34.688517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ee47c-df91-42c1-9d66-47a5ddc3eeb1 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:34.693127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13d7d9-3060-472b-be80-c3d1ff1a1801 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:34.703008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 154/310] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Aug 10 09:50:34 2022] PUT /identity/v3/system/users/b9925d98e992405e8e8dd35edbc3de67/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:34.703008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4823da5-07ce-4011-8ae1-cdb690d52b6e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.703008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4823da5-07ce-4011-8ae1-cdb690d52b6e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.703008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4823da5-07ce-4011-8ae1-cdb690d52b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.731437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eededf-7d2d-4a88-969f-e857807dbfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:34.732037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e4823da5-07ce-4011-8ae1-cdb690d52b6e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:34.732145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eededf-7d2d-4a88-969f-e857807dbfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:34.732355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eededf-7d2d-4a88-969f-e857807dbfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:34.732668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eededf-7d2d-4a88-969f-e857807dbfa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:34.733390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e4823da5-07ce-4011-8ae1-cdb690d52b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:34.733497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eededf-7d2d-4a88-969f-e857807dbfa6 None admin] RBAC: Adding query filter params (role.id=053f81c391444db1b6409c4eb953dc35, user.id=04ab4ca23e7a4bfdb4bb717732648e44, scope.system=all) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:34.733692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eededf-7d2d-4a88-969f-e857807dbfa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:34.735236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eededf-7d2d-4a88-969f-e857807dbfa6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:34.741344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 157/311] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 10 09:50:34 2022] GET /identity/v3/role_assignments?user.id=04ab4ca23e7a4bfdb4bb717732648e44&scope.system=all&role.id=053f81c391444db1b6409c4eb953dc35 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:34.762693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:34.823258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 155/312] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:34.846307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eededf-7d2d-4a88-969f-e857807dbfa6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:34.940383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:34.965143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eededf-7d2d-4a88-969f-e857807dbfa6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:35.000752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2eb054-3563-4699-921c-0e836fc3c130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:35.001406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eb054-3563-4699-921c-0e836fc3c130 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:35.001682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eb054-3563-4699-921c-0e836fc3c130 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:35.001950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eb054-3563-4699-921c-0e836fc3c130 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:35.014133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2eb054-3563-4699-921c-0e836fc3c130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6a2eb054-3563-4699-921c-0e836fc3c130 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:35.019376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:35.020225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:35.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 158/313] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:35.027404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4823da5-07ce-4011-8ae1-cdb690d52b6e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:35.135739 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4823da5-07ce-4011-8ae1-cdb690d52b6e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:35.171845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467feda-b875-4596-b6c2-6142f5c34526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:35.172816 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467feda-b875-4596-b6c2-6142f5c34526 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:35.173310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467feda-b875-4596-b6c2-6142f5c34526 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:35.173776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467feda-b875-4596-b6c2-6142f5c34526 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:35.174626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467feda-b875-4596-b6c2-6142f5c34526 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:35.175162 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467feda-b875-4596-b6c2-6142f5c34526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:35.176737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467feda-b875-4596-b6c2-6142f5c34526 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:35.188331 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 156/314] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:35.193533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2eb054-3563-4699-921c-0e836fc3c130 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:35.297474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2eb054-3563-4699-921c-0e836fc3c130 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:35.331851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a64f6d-1dd5-4689-a861-dc4458efeeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:35.332475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a64f6d-1dd5-4689-a861-dc4458efeeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:35.332749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a64f6d-1dd5-4689-a861-dc4458efeeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:35.333018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a64f6d-1dd5-4689-a861-dc4458efeeb3 None admin] PATH_INFO: `/v3/users/002fd00099394087b24dc6d26ef9fa9a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:35.334252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a64f6d-1dd5-4689-a861-dc4458efeeb3 None admin] RBAC: Authorizing `identity:get_user(user_id=002fd00099394087b24dc6d26ef9fa9a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:35.335719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a64f6d-1dd5-4689-a861-dc4458efeeb3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:35.337613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 159/315] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:35 2022] GET /identity/v3/users/002fd00099394087b24dc6d26ef9fa9a => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:35.343084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467feda-b875-4596-b6c2-6142f5c34526 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:35.369623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:35.464123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467feda-b875-4596-b6c2-6142f5c34526 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:35.499478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85971f-e21a-45dc-b5fd-bd9d55a77c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:35.500425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85971f-e21a-45dc-b5fd-bd9d55a77c4a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:35.500908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85971f-e21a-45dc-b5fd-bd9d55a77c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:35.501375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85971f-e21a-45dc-b5fd-bd9d55a77c4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:35.571250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85971f-e21a-45dc-b5fd-bd9d55a77c4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:35.573470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85971f-e21a-45dc-b5fd-bd9d55a77c4a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:35.596844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:35.673622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 157/316] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:35.679508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a64f6d-1dd5-4689-a861-dc4458efeeb3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:35.699006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:35.790823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a64f6d-1dd5-4689-a861-dc4458efeeb3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:35.826307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52299955-dbc4-4ed2-8abb-2ac6a52a49dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:35.827308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52299955-dbc4-4ed2-8abb-2ac6a52a49dd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:35.827788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52299955-dbc4-4ed2-8abb-2ac6a52a49dd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:35.828255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52299955-dbc4-4ed2-8abb-2ac6a52a49dd None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:35.829599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52299955-dbc4-4ed2-8abb-2ac6a52a49dd None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:35.831256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52299955-dbc4-4ed2-8abb-2ac6a52a49dd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:35.833401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 160/317] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:35 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:35.839526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85971f-e21a-45dc-b5fd-bd9d55a77c4a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:35.912498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850c3a5-e961-4047-b75d-4dc73a63363f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:35.912819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850c3a5-e961-4047-b75d-4dc73a63363f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:35.913051 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850c3a5-e961-4047-b75d-4dc73a63363f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:35.914578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 161/318] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:50:35.919898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f815f-133a-4a2c-85f4-0b419715ef87 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:35.920178 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f815f-133a-4a2c-85f4-0b419715ef87 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:35.920505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f815f-133a-4a2c-85f4-0b419715ef87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:35.943915 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9f6f815f-133a-4a2c-85f4-0b419715ef87 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:35.945025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f6f815f-133a-4a2c-85f4-0b419715ef87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:35.951513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85971f-e21a-45dc-b5fd-bd9d55a77c4a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:35.975516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:35.990108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba1b13-311f-4c32-856e-bb9ac5b8e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:35.991175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba1b13-311f-4c32-856e-bb9ac5b8e861 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:35.991704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba1b13-311f-4c32-856e-bb9ac5b8e861 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:35.992176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba1b13-311f-4c32-856e-bb9ac5b8e861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:35.993418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba1b13-311f-4c32-856e-bb9ac5b8e861 None admin] RBAC: Adding query filter params (role.id=053f81c391444db1b6409c4eb953dc35, scope.project.id=d254037584f44b54b09c2b3e76807918, user.id=002fd00099394087b24dc6d26ef9fa9a) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:35.993946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba1b13-311f-4c32-856e-bb9ac5b8e861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:35.995742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba1b13-311f-4c32-856e-bb9ac5b8e861 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:36.002485 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 158/319] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:50:35 2022] GET /identity/v3/role_assignments?user.id=002fd00099394087b24dc6d26ef9fa9a&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=053f81c391444db1b6409c4eb953dc35 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:36.015636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 162/320] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:36.039039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba1b13-311f-4c32-856e-bb9ac5b8e861 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:36.151953 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba1b13-311f-4c32-856e-bb9ac5b8e861 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:36.187184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0888a6-a070-470e-bece-a59898171a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:36.188041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0888a6-a070-470e-bece-a59898171a24 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:36.188461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0888a6-a070-470e-bece-a59898171a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:36.188870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0888a6-a070-470e-bece-a59898171a24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:36.190130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0888a6-a070-470e-bece-a59898171a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:36.191815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0888a6-a070-470e-bece-a59898171a24 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:36.199147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 159/321] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:36.200683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f815f-133a-4a2c-85f4-0b419715ef87 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:36.203869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:36.307854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f815f-133a-4a2c-85f4-0b419715ef87 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:36.341846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea2cbe-e8a9-45be-b135-cc5c71368812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:36.342457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea2cbe-e8a9-45be-b135-cc5c71368812 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:36.342734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea2cbe-e8a9-45be-b135-cc5c71368812 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:36.343034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea2cbe-e8a9-45be-b135-cc5c71368812 None admin] PATH_INFO: `/v3/groups` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:36.343671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea2cbe-e8a9-45be-b135-cc5c71368812 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:36.345092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea2cbe-e8a9-45be-b135-cc5c71368812 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:36.363474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 163/322] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Aug 10 09:50:36 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:36.788089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56705bd-199a-44a3-af69-eef9e5c0ec8c None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:36.788600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56705bd-199a-44a3-af69-eef9e5c0ec8c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:36.788600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56705bd-199a-44a3-af69-eef9e5c0ec8c None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:36.790721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 160/323] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:36.796182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94fe2-c68b-4b45-9a8b-4832417415ab None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:36.796619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94fe2-c68b-4b45-9a8b-4832417415ab None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:36.796809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94fe2-c68b-4b45-9a8b-4832417415ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:36.818003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b6d94fe2-c68b-4b45-9a8b-4832417415ab None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:36.819213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b6d94fe2-c68b-4b45-9a8b-4832417415ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:36.860598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36910a-6e6c-4260-8eb6-3993fd2b2036 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:36.860879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36910a-6e6c-4260-8eb6-3993fd2b2036 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:36.861346 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36910a-6e6c-4260-8eb6-3993fd2b2036 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:36.862824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 161/324] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:36.869226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb983c0-2490-4086-9963-dcc06f462769 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:36.869983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb983c0-2490-4086-9963-dcc06f462769 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:36.870479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb983c0-2490-4086-9963-dcc06f462769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:36.892871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0cb983c0-2490-4086-9963-dcc06f462769 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:36.894703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0cb983c0-2490-4086-9963-dcc06f462769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:36.899564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:36.913882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 164/325] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:36.936683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d94fe2-c68b-4b45-9a8b-4832417415ab None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:36.970263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:36.998105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 162/326] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:37.020011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb983c0-2490-4086-9963-dcc06f462769 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:37.070577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d94fe2-c68b-4b45-9a8b-4832417415ab None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:37.112867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480db37-0122-4c5c-ada8-a80c86b7a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:37.113744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480db37-0122-4c5c-ada8-a80c86b7a80e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:37.114170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480db37-0122-4c5c-ada8-a80c86b7a80e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:37.114626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480db37-0122-4c5c-ada8-a80c86b7a80e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:37.128031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480db37-0122-4c5c-ada8-a80c86b7a80e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6480db37-0122-4c5c-ada8-a80c86b7a80e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:37.137240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:37.138171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:37.138992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 165/327] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:37.145110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480db37-0122-4c5c-ada8-a80c86b7a80e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:37.156125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb983c0-2490-4086-9963-dcc06f462769 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:37.196128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4a66b-6138-430e-a96c-376c47bc3a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:37.197265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4a66b-6138-430e-a96c-376c47bc3a80 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:37.197546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4a66b-6138-430e-a96c-376c47bc3a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:37.197866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4a66b-6138-430e-a96c-376c47bc3a80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:37.210510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4a66b-6138-430e-a96c-376c47bc3a80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-0ed4a66b-6138-430e-a96c-376c47bc3a80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:37.216657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:37.217520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:37.218318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 163/328] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:37.224026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4a66b-6138-430e-a96c-376c47bc3a80 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:37.267252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:37.278552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:37.336930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4a66b-6138-430e-a96c-376c47bc3a80 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:37.339326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480db37-0122-4c5c-ada8-a80c86b7a80e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:37.373971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3bf5-3da0-4a98-a3b9-eb4480cec806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:37.374581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3bf5-3da0-4a98-a3b9-eb4480cec806 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:37.374864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3bf5-3da0-4a98-a3b9-eb4480cec806 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:37.376309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3bf5-3da0-4a98-a3b9-eb4480cec806 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:37.377096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c3bf5-3da0-4a98-a3b9-eb4480cec806 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:37.377408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c3bf5-3da0-4a98-a3b9-eb4480cec806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:37.378810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c3bf5-3da0-4a98-a3b9-eb4480cec806 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:37.388365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 164/329] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:37.394161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3bf5-3da0-4a98-a3b9-eb4480cec806 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:37.404581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223a471-f2d1-4ab8-8fd7-5b4fd95ff780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:37.404581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223a471-f2d1-4ab8-8fd7-5b4fd95ff780 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:37.404581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223a471-f2d1-4ab8-8fd7-5b4fd95ff780 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:37.404581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223a471-f2d1-4ab8-8fd7-5b4fd95ff780 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:37.404958 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223a471-f2d1-4ab8-8fd7-5b4fd95ff780 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:37.404958 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223a471-f2d1-4ab8-8fd7-5b4fd95ff780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:37.406087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223a471-f2d1-4ab8-8fd7-5b4fd95ff780 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:37.419251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 166/330] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:37.427944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223a471-f2d1-4ab8-8fd7-5b4fd95ff780 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:37.516728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3bf5-3da0-4a98-a3b9-eb4480cec806 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:37.524453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:37.551716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7595f6-363c-4be1-aae6-2e7de11742b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:37.555310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7595f6-363c-4be1-aae6-2e7de11742b7 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:37.555310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7595f6-363c-4be1-aae6-2e7de11742b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:37.555310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7595f6-363c-4be1-aae6-2e7de11742b7 None admin] PATH_INFO: `/v3/users/04ab4ca23e7a4bfdb4bb717732648e44` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:37.555310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7595f6-363c-4be1-aae6-2e7de11742b7 None admin] RBAC: Authorizing `identity:get_user(user_id=04ab4ca23e7a4bfdb4bb717732648e44)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:37.556158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7595f6-363c-4be1-aae6-2e7de11742b7 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:37.558036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 165/331] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:37 2022] GET /identity/v3/users/04ab4ca23e7a4bfdb4bb717732648e44 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:37.563634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7595f6-363c-4be1-aae6-2e7de11742b7 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:37.620107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223a471-f2d1-4ab8-8fd7-5b4fd95ff780 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:37.641393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:37.677169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd5668-f7ac-4a94-942b-6f582b5d5b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:37.678152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd5668-f7ac-4a94-942b-6f582b5d5b83 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:37.678656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd5668-f7ac-4a94-942b-6f582b5d5b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:37.679153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd5668-f7ac-4a94-942b-6f582b5d5b83 None admin] PATH_INFO: `/v3/users/b9925d98e992405e8e8dd35edbc3de67` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:37.682360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd5668-f7ac-4a94-942b-6f582b5d5b83 None admin] RBAC: Authorizing `identity:get_user(user_id=b9925d98e992405e8e8dd35edbc3de67)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:37.684054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd5668-f7ac-4a94-942b-6f582b5d5b83 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:37.690804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 167/332] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:37 2022] GET /identity/v3/users/b9925d98e992405e8e8dd35edbc3de67 => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:37.698094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd5668-f7ac-4a94-942b-6f582b5d5b83 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:37.743990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7595f6-363c-4be1-aae6-2e7de11742b7 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:37.803471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c838f8-d2c2-430c-a4a9-074dbb4b4d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:37.804171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c838f8-d2c2-430c-a4a9-074dbb4b4d01 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:37.804407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c838f8-d2c2-430c-a4a9-074dbb4b4d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:37.804684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c838f8-d2c2-430c-a4a9-074dbb4b4d01 None admin] PATH_INFO: `/v3/system/users/04ab4ca23e7a4bfdb4bb717732648e44/roles/053f81c391444db1b6409c4eb953dc35` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:37.806812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c838f8-d2c2-430c-a4a9-074dbb4b4d01 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=04ab4ca23e7a4bfdb4bb717732648e44, role_id=053f81c391444db1b6409c4eb953dc35)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:37.808221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c838f8-d2c2-430c-a4a9-074dbb4b4d01 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:37.826590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 166/333] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Aug 10 09:50:37 2022] PUT /identity/v3/system/users/04ab4ca23e7a4bfdb4bb717732648e44/roles/053f81c391444db1b6409c4eb953dc35 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:37.860082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd5668-f7ac-4a94-942b-6f582b5d5b83 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:37.899025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095b07f-fd1d-40f5-b338-59081690838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:37.899553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095b07f-fd1d-40f5-b338-59081690838a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:37.899831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095b07f-fd1d-40f5-b338-59081690838a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:37.900101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095b07f-fd1d-40f5-b338-59081690838a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:37.901982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095b07f-fd1d-40f5-b338-59081690838a None admin] RBAC: Adding query filter params (role.id=7694dcfc7d6547abb6565cb9e726eacd, user.id=b9925d98e992405e8e8dd35edbc3de67, scope.system=all) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:37.902307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095b07f-fd1d-40f5-b338-59081690838a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:37.903832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095b07f-fd1d-40f5-b338-59081690838a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:37.918710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 168/334] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 10 09:50:37 2022] GET /identity/v3/role_assignments?user.id=b9925d98e992405e8e8dd35edbc3de67&scope.system=all&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:38.948111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1abc7a-fb1d-4807-a6a1-d4c376c76dbd None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:38.948620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1abc7a-fb1d-4807-a6a1-d4c376c76dbd None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:38.948620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1abc7a-fb1d-4807-a6a1-d4c376c76dbd None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:38.950190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 167/335] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:38.958253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99a1d5-04dd-4751-9c1f-481be8088487 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:38.959153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99a1d5-04dd-4751-9c1f-481be8088487 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:38.959641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99a1d5-04dd-4751-9c1f-481be8088487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:38.987910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9e99a1d5-04dd-4751-9c1f-481be8088487 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:38.989394 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9e99a1d5-04dd-4751-9c1f-481be8088487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:39.031127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:39.072669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 169/336] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:39.114261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1abc7a-fb1d-4807-a6a1-d4c376c76dbd None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:39.158175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:39.161710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3061e3-97cb-41ff-8c62-9ab857031e39 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:39.161989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3061e3-97cb-41ff-8c62-9ab857031e39 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:39.162320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3061e3-97cb-41ff-8c62-9ab857031e39 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:39.163692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 170/337] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:50:39.169589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940fb79-2d08-4bb9-8459-5b051fbb38a8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:39.169876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940fb79-2d08-4bb9-8459-5b051fbb38a8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:39.170152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940fb79-2d08-4bb9-8459-5b051fbb38a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:39.191315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e940fb79-2d08-4bb9-8459-5b051fbb38a8 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:39.192735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e940fb79-2d08-4bb9-8459-5b051fbb38a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:39.263289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1abc7a-fb1d-4807-a6a1-d4c376c76dbd None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:39.303062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 171/338] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:39.323635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ce496-af24-4267-9833-59f3723ac66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:39.324520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ce496-af24-4267-9833-59f3723ac66b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:39.324944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ce496-af24-4267-9833-59f3723ac66b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:39.325371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ce496-af24-4267-9833-59f3723ac66b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:39.340056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ce496-af24-4267-9833-59f3723ac66b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-3f3ce496-af24-4267-9833-59f3723ac66b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:39.348336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:39.349399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:39.350344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:39.352737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 168/339] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:39.358313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940fb79-2d08-4bb9-8459-5b051fbb38a8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:39.369090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ce496-af24-4267-9833-59f3723ac66b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:39.487352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ce496-af24-4267-9833-59f3723ac66b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:39.491122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:39.526774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdee79-d4bc-473e-9dcf-6815b4b8cf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:39.527813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdee79-d4bc-473e-9dcf-6815b4b8cf1c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:39.528235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdee79-d4bc-473e-9dcf-6815b4b8cf1c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:39.528646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdee79-d4bc-473e-9dcf-6815b4b8cf1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:39.529133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940fb79-2d08-4bb9-8459-5b051fbb38a8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:39.531245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:39.542507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdee79-d4bc-473e-9dcf-6815b4b8cf1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-53fdee79-d4bc-473e-9dcf-6815b4b8cf1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:39.547495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:39.548512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:39.549504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:39.552192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 169/340] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:39.560139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdee79-d4bc-473e-9dcf-6815b4b8cf1c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:39.631663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a51f3-8b9d-4530-aa0b-367e35d06fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:39.632286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a51f3-8b9d-4530-aa0b-367e35d06fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:39.632562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a51f3-8b9d-4530-aa0b-367e35d06fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:39.632828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a51f3-8b9d-4530-aa0b-367e35d06fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:39.633542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a51f3-8b9d-4530-aa0b-367e35d06fc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:39.633841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a51f3-8b9d-4530-aa0b-367e35d06fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:39.635240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a51f3-8b9d-4530-aa0b-367e35d06fc2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:39.654069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 172/341] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:39.663956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a51f3-8b9d-4530-aa0b-367e35d06fc2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:39.688734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdee79-d4bc-473e-9dcf-6815b4b8cf1c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:39.726309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5c721-3839-4e44-8d6a-104ee7b31455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:39.727747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5c721-3839-4e44-8d6a-104ee7b31455 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:39.728172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5c721-3839-4e44-8d6a-104ee7b31455 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:39.728936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5c721-3839-4e44-8d6a-104ee7b31455 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:39.729917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5c721-3839-4e44-8d6a-104ee7b31455 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:39.730403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5c721-3839-4e44-8d6a-104ee7b31455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:39.731977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5c721-3839-4e44-8d6a-104ee7b31455 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:39.741202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 170/342] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:39.748914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5c721-3839-4e44-8d6a-104ee7b31455 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:39.784708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:39.867491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5c721-3839-4e44-8d6a-104ee7b31455 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:39.931646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e520f-c50d-465c-b0d1-605e138baf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:39.932505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e520f-c50d-465c-b0d1-605e138baf21 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:39.932931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e520f-c50d-465c-b0d1-605e138baf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:39.933347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e520f-c50d-465c-b0d1-605e138baf21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:39.937089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a51f3-8b9d-4530-aa0b-367e35d06fc2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:40.025467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e520f-c50d-465c-b0d1-605e138baf21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:40.027635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e520f-c50d-465c-b0d1-605e138baf21 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:40.051681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23997-43f9-4758-8f5a-f8acd0d57e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:40.052313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23997-43f9-4758-8f5a-f8acd0d57e08 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:40.052770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23997-43f9-4758-8f5a-f8acd0d57e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:40.053176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23997-43f9-4758-8f5a-f8acd0d57e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:40.067114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:40.159972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:40.166429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 171/343] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:40.178360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e520f-c50d-465c-b0d1-605e138baf21 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:40.225267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23997-43f9-4758-8f5a-f8acd0d57e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:40.226749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23997-43f9-4758-8f5a-f8acd0d57e08 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:40.321301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e520f-c50d-465c-b0d1-605e138baf21 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:40.342687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:40.358354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ad73b-c7de-451b-873c-2b1f7ca48485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:40.359278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ad73b-c7de-451b-873c-2b1f7ca48485 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:40.359780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ad73b-c7de-451b-873c-2b1f7ca48485 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:40.360269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ad73b-c7de-451b-873c-2b1f7ca48485 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:40.364807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ad73b-c7de-451b-873c-2b1f7ca48485 None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:40.366475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ad73b-c7de-451b-873c-2b1f7ca48485 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:40.368967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 172/344] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:40 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:40.370418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7f4b0-c21a-41f1-be17-3071d09a69ab None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:40.370900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7f4b0-c21a-41f1-be17-3071d09a69ab None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:40.371412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7f4b0-c21a-41f1-be17-3071d09a69ab None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:40.372828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 173/345] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:40.380289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fa4a1-d9f0-464e-9380-fc24d4cf554f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:40.380566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fa4a1-d9f0-464e-9380-fc24d4cf554f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:40.380837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fa4a1-d9f0-464e-9380-fc24d4cf554f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:40.410461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-250fa4a1-d9f0-464e-9380-fc24d4cf554f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:40.410461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-250fa4a1-d9f0-464e-9380-fc24d4cf554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:40.418177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 173/346] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 09:50:40.418865 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23997-43f9-4758-8f5a-f8acd0d57e08 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:40.495235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:40.541943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23997-43f9-4758-8f5a-f8acd0d57e08 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:40.580335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 174/347] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:40.581895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fa4a1-d9f0-464e-9380-fc24d4cf554f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:40.585004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9eae2-5ff9-4aa8-809e-343b0bae1c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:40.585704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9eae2-5ff9-4aa8-809e-343b0bae1c36 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:40.586024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9eae2-5ff9-4aa8-809e-343b0bae1c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:40.586811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9eae2-5ff9-4aa8-809e-343b0bae1c36 None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:40.598909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9eae2-5ff9-4aa8-809e-343b0bae1c36 None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:40.600391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9eae2-5ff9-4aa8-809e-343b0bae1c36 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:40.602890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 174/348] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:40 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:40.609781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9eae2-5ff9-4aa8-809e-343b0bae1c36 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:40.766785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9eae2-5ff9-4aa8-809e-343b0bae1c36 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:40.797354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:40.806763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fa4a1-d9f0-464e-9380-fc24d4cf554f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:40.809191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8f644-6129-44e9-a11a-06f0389d57b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:40.809799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8f644-6129-44e9-a11a-06f0389d57b0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:40.810135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8f644-6129-44e9-a11a-06f0389d57b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:40.810409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8f644-6129-44e9-a11a-06f0389d57b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:40.811159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8f644-6129-44e9-a11a-06f0389d57b0 None admin] RBAC: Adding query filter params (group.id=01363f1eb6934ebf88c6f0b95ff1bb3c, role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:40.811462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8f644-6129-44e9-a11a-06f0389d57b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:40.813947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8f644-6129-44e9-a11a-06f0389d57b0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:40.820298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 175/349] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:40 2022] GET /identity/v3/role_assignments?group.id=01363f1eb6934ebf88c6f0b95ff1bb3c&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:40.821197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8f644-6129-44e9-a11a-06f0389d57b0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:40.848379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:40.890504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82058575-61d7-4fcf-bdca-c630d7752554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:40.891564 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82058575-61d7-4fcf-bdca-c630d7752554 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:40.892048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82058575-61d7-4fcf-bdca-c630d7752554 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:40.892518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82058575-61d7-4fcf-bdca-c630d7752554 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:40.893867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82058575-61d7-4fcf-bdca-c630d7752554 None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:40.900866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82058575-61d7-4fcf-bdca-c630d7752554 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:40.902936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 175/350] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:40 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:40.924561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82058575-61d7-4fcf-bdca-c630d7752554 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:40.947735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8f644-6129-44e9-a11a-06f0389d57b0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:40.987225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3efcf-a9cf-48ab-b5ac-3eb82b9ae020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:40.988274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3efcf-a9cf-48ab-b5ac-3eb82b9ae020 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:40.988694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3efcf-a9cf-48ab-b5ac-3eb82b9ae020 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:40.989100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3efcf-a9cf-48ab-b5ac-3eb82b9ae020 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.002636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3efcf-a9cf-48ab-b5ac-3eb82b9ae020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-20b3efcf-a9cf-48ab-b5ac-3eb82b9ae020 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:41.009613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:41.010537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:41.011622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:41.014135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 176/351] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:41.020535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3efcf-a9cf-48ab-b5ac-3eb82b9ae020 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:41.085318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:41.153825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82058575-61d7-4fcf-bdca-c630d7752554 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:41.160207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3efcf-a9cf-48ab-b5ac-3eb82b9ae020 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:41.203629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f11e7-9781-4cce-b747-904844f91912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:41.204276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f11e7-9781-4cce-b747-904844f91912 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.204553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f11e7-9781-4cce-b747-904844f91912 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.204825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f11e7-9781-4cce-b747-904844f91912 None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.204927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd7cd5-3435-4081-bc45-7815152c6039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:41.205606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd7cd5-3435-4081-bc45-7815152c6039 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.205946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f11e7-9781-4cce-b747-904844f91912 None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:41.206213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd7cd5-3435-4081-bc45-7815152c6039 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.206629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd7cd5-3435-4081-bc45-7815152c6039 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.207417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f11e7-9781-4cce-b747-904844f91912 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:41.207645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd7cd5-3435-4081-bc45-7815152c6039 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:41.208106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd7cd5-3435-4081-bc45-7815152c6039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:41.209323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 176/352] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:40 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:41.209976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd7cd5-3435-4081-bc45-7815152c6039 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:41.210618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc063a-36ca-4e03-ae45-33876e85f765 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.210898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc063a-36ca-4e03-ae45-33876e85f765 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.211751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc063a-36ca-4e03-ae45-33876e85f765 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.212932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 177/353] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:41.214083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d48ba-a520-489e-898e-0011df4abb5d None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.214551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d48ba-a520-489e-898e-0011df4abb5d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.214997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d48ba-a520-489e-898e-0011df4abb5d None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.216583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 178/354] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:41.217439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d48ba-a520-489e-898e-0011df4abb5d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:41.233193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 177/355] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:41.234610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1913be-6284-43df-ad7b-8e677771d5c3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.234899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1913be-6284-43df-ad7b-8e677771d5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.238973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1913be-6284-43df-ad7b-8e677771d5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.267802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bf1913be-6284-43df-ad7b-8e677771d5c3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:41.268998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bf1913be-6284-43df-ad7b-8e677771d5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:41.282721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:41.337038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d48ba-a520-489e-898e-0011df4abb5d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:41.348399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 178/356] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:41.349974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b7fd6-af00-44a9-9af3-ee6b4c08639d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.350439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b7fd6-af00-44a9-9af3-ee6b4c08639d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.350775 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b7fd6-af00-44a9-9af3-ee6b4c08639d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.372717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-287b7fd6-af00-44a9-9af3-ee6b4c08639d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:41.374181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-287b7fd6-af00-44a9-9af3-ee6b4c08639d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:41.386328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d5b09-af4d-4bb5-bd35-29878bda3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:41.387192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d5b09-af4d-4bb5-bd35-29878bda3c4a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.387612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d5b09-af4d-4bb5-bd35-29878bda3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.388024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d5b09-af4d-4bb5-bd35-29878bda3c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.389030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d5b09-af4d-4bb5-bd35-29878bda3c4a None admin] RBAC: Adding query filter params (group.id=01363f1eb6934ebf88c6f0b95ff1bb3c, role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:41.389527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d5b09-af4d-4bb5-bd35-29878bda3c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:41.391603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d5b09-af4d-4bb5-bd35-29878bda3c4a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:41.402283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 179/357] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:41 2022] GET /identity/v3/role_assignments?group.id=01363f1eb6934ebf88c6f0b95ff1bb3c&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:41.403556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d5b09-af4d-4bb5-bd35-29878bda3c4a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:41.426558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:41.466054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 179/358] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:41.467374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ebd0-687e-44d1-98ca-27d2bb851c4a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.467659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ebd0-687e-44d1-98ca-27d2bb851c4a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.467924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ebd0-687e-44d1-98ca-27d2bb851c4a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.469505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 180/359] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:41.470177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ebd0-687e-44d1-98ca-27d2bb851c4a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:41.563245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d5b09-af4d-4bb5-bd35-29878bda3c4a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:41.585186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:41.605273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f2016-9e0f-4629-a29c-4af1be77bfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:41.606145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f2016-9e0f-4629-a29c-4af1be77bfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.606708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f2016-9e0f-4629-a29c-4af1be77bfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.607006 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f2016-9e0f-4629-a29c-4af1be77bfe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.618447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ebd0-687e-44d1-98ca-27d2bb851c4a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:41.664741 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13fbcd-449a-4ab9-907d-5033eb53e2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:41.665838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13fbcd-449a-4ab9-907d-5033eb53e2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.666297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13fbcd-449a-4ab9-907d-5033eb53e2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.667143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13fbcd-449a-4ab9-907d-5033eb53e2b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.684576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13fbcd-449a-4ab9-907d-5033eb53e2b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0f13fbcd-449a-4ab9-907d-5033eb53e2b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:41.691557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:41.692477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:41.693250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:41.696804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 181/360] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:50:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:41.698306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11878a1-173d-41b9-bab9-51028b2dbdde None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:41.698783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11878a1-173d-41b9-bab9-51028b2dbdde None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:41.699209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11878a1-173d-41b9-bab9-51028b2dbdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:41.702999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f2016-9e0f-4629-a29c-4af1be77bfe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:41.705847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f2016-9e0f-4629-a29c-4af1be77bfe3 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:41.723117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a11878a1-173d-41b9-bab9-51028b2dbdde None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:41.724687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a11878a1-173d-41b9-bab9-51028b2dbdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:41.763628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:41.820719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 182/361] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:41.822370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11878a1-173d-41b9-bab9-51028b2dbdde None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:41.855391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 180/362] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:41.856249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f2016-9e0f-4629-a29c-4af1be77bfe3 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:41.881146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:41.959013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11878a1-173d-41b9-bab9-51028b2dbdde None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:41.973670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f2016-9e0f-4629-a29c-4af1be77bfe3 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.005576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9c282-bf21-43a1-a58f-b0883a353764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.006219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9c282-bf21-43a1-a58f-b0883a353764 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.006494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9c282-bf21-43a1-a58f-b0883a353764 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.006761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9c282-bf21-43a1-a58f-b0883a353764 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.019444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9c282-bf21-43a1-a58f-b0883a353764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-7ca9c282-bf21-43a1-a58f-b0883a353764 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:42.026191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:42.027154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 09:50:42.027959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:42.028561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4d104-1633-412d-92a2-830454ef2b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.028561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4d104-1633-412d-92a2-830454ef2b7f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.028707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 183/363] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:42.028773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4d104-1633-412d-92a2-830454ef2b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.028773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4d104-1633-412d-92a2-830454ef2b7f None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.028886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9c282-bf21-43a1-a58f-b0883a353764 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:42.028949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4d104-1633-412d-92a2-830454ef2b7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:42.028949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4d104-1633-412d-92a2-830454ef2b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:42.034975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4d104-1633-412d-92a2-830454ef2b7f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:42.041050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 181/364] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:42.041901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4d104-1633-412d-92a2-830454ef2b7f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:42.069826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:42.158307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:42.159294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9c282-bf21-43a1-a58f-b0883a353764 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.184882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4d104-1633-412d-92a2-830454ef2b7f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.204974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fea485-d563-4f4a-b58e-5562cec475cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.205876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fea485-d563-4f4a-b58e-5562cec475cc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.206323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fea485-d563-4f4a-b58e-5562cec475cc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.206788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fea485-d563-4f4a-b58e-5562cec475cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.219449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fea485-d563-4f4a-b58e-5562cec475cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f5fea485-d563-4f4a-b58e-5562cec475cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:42.227449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:42.228337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:42.229103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:42.231103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 184/365] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:42.232224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fea485-d563-4f4a-b58e-5562cec475cc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:42.239907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1d567-d9d9-4fa8-ae28-7f701795c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.240790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1d567-d9d9-4fa8-ae28-7f701795c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.241072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1d567-d9d9-4fa8-ae28-7f701795c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.241346 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1d567-d9d9-4fa8-ae28-7f701795c4f2 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.242591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1d567-d9d9-4fa8-ae28-7f701795c4f2 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:42.244514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1d567-d9d9-4fa8-ae28-7f701795c4f2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:42.246189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 182/366] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:42 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:42.247112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1d567-d9d9-4fa8-ae28-7f701795c4f2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:42.340776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:42.358266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fea485-d563-4f4a-b58e-5562cec475cc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.371779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1d567-d9d9-4fa8-ae28-7f701795c4f2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.395972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025baad0-d720-4ae4-8127-1f92aa93779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.396907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025baad0-d720-4ae4-8127-1f92aa93779f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.397332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025baad0-d720-4ae4-8127-1f92aa93779f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.397781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025baad0-d720-4ae4-8127-1f92aa93779f None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.399169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025baad0-d720-4ae4-8127-1f92aa93779f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:42.399617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025baad0-d720-4ae4-8127-1f92aa93779f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:42.401304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025baad0-d720-4ae4-8127-1f92aa93779f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:42.410521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 185/367] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:42.411762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a5fe3-5484-4433-a368-4c36d2e5644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.412186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025baad0-d720-4ae4-8127-1f92aa93779f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:42.412526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5fe3-5484-4433-a368-4c36d2e5644a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.412966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5fe3-5484-4433-a368-4c36d2e5644a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.413396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5fe3-5484-4433-a368-4c36d2e5644a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.469259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:42.548840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a5fe3-5484-4433-a368-4c36d2e5644a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:42.551100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a5fe3-5484-4433-a368-4c36d2e5644a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:42.554695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025baad0-d720-4ae4-8127-1f92aa93779f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.595066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f848d5-89d7-4b21-9148-62f644a21746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.595942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f848d5-89d7-4b21-9148-62f644a21746 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.596358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f848d5-89d7-4b21-9148-62f644a21746 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.596769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f848d5-89d7-4b21-9148-62f644a21746 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.597625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f848d5-89d7-4b21-9148-62f644a21746 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:42.598131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f848d5-89d7-4b21-9148-62f644a21746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:42.599784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f848d5-89d7-4b21-9148-62f644a21746 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:42.611477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 186/368] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:42.618304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f848d5-89d7-4b21-9148-62f644a21746 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:42.630142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:42.687110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 183/369] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:42.688547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a5fe3-5484-4433-a368-4c36d2e5644a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:42.747694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:42.750986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f848d5-89d7-4b21-9148-62f644a21746 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.791759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7371a-94fd-41bc-a38b-93a6f4756160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.792740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7371a-94fd-41bc-a38b-93a6f4756160 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.793236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7371a-94fd-41bc-a38b-93a6f4756160 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.793788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7371a-94fd-41bc-a38b-93a6f4756160 None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.795180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7371a-94fd-41bc-a38b-93a6f4756160 None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:42.797460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7371a-94fd-41bc-a38b-93a6f4756160 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:42.799606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 187/370] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:42 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:42.800555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7371a-94fd-41bc-a38b-93a6f4756160 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:42.812430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a5fe3-5484-4433-a368-4c36d2e5644a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.849329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1970c3-0c2f-4ea9-8e27-a40826c7e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.849967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1970c3-0c2f-4ea9-8e27-a40826c7e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.850246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1970c3-0c2f-4ea9-8e27-a40826c7e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.850519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1970c3-0c2f-4ea9-8e27-a40826c7e4a4 None admin] PATH_INFO: `/v3/users/04ab4ca23e7a4bfdb4bb717732648e44` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.852290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1970c3-0c2f-4ea9-8e27-a40826c7e4a4 None admin] RBAC: Authorizing `identity:get_user(user_id=04ab4ca23e7a4bfdb4bb717732648e44)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:42.853915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1970c3-0c2f-4ea9-8e27-a40826c7e4a4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:42.855985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 184/371] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:50:42 2022] GET /identity/v3/users/04ab4ca23e7a4bfdb4bb717732648e44 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:42.856889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1970c3-0c2f-4ea9-8e27-a40826c7e4a4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:42.906441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:42.933979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7371a-94fd-41bc-a38b-93a6f4756160 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.965358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1970c3-0c2f-4ea9-8e27-a40826c7e4a4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:42.972466 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f681d-dc82-4e06-abe2-b873a4e2fe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:42.973367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f681d-dc82-4e06-abe2-b873a4e2fe8d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:42.974309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f681d-dc82-4e06-abe2-b873a4e2fe8d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:42.974799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f681d-dc82-4e06-abe2-b873a4e2fe8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:42.986396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:43.007184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae6c26-1caa-466a-9855-636b55a6e07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:43.007806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae6c26-1caa-466a-9855-636b55a6e07a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.008280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae6c26-1caa-466a-9855-636b55a6e07a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.008719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae6c26-1caa-466a-9855-636b55a6e07a None admin] PATH_INFO: `/v3/projects/9cee49555c8143ed868a4be75b65ed07` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.010193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae6c26-1caa-466a-9855-636b55a6e07a None admin] RBAC: Authorizing `identity:get_project(project_id=9cee49555c8143ed868a4be75b65ed07)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:43.012838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae6c26-1caa-466a-9855-636b55a6e07a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:43.019057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 185/372] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:42 2022] GET /identity/v3/projects/9cee49555c8143ed868a4be75b65ed07 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:43.020154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae6c26-1caa-466a-9855-636b55a6e07a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:43.052322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f681d-dc82-4e06-abe2-b873a4e2fe8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:43.053898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f681d-dc82-4e06-abe2-b873a4e2fe8d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:43.129436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae6c26-1caa-466a-9855-636b55a6e07a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:43.136747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:43.162483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 188/373] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:43.164212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f681d-dc82-4e06-abe2-b873a4e2fe8d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:43.172142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1af92-9d44-4293-8335-b13e06be1fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:43.172752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1af92-9d44-4293-8335-b13e06be1fbf None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.173194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1af92-9d44-4293-8335-b13e06be1fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.173611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1af92-9d44-4293-8335-b13e06be1fbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.174533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1af92-9d44-4293-8335-b13e06be1fbf None admin] RBAC: Adding query filter params (group.id=01363f1eb6934ebf88c6f0b95ff1bb3c, role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=d254037584f44b54b09c2b3e76807918) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:43.175147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1af92-9d44-4293-8335-b13e06be1fbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:43.176879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1af92-9d44-4293-8335-b13e06be1fbf None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:43.183986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 186/374] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:43 2022] GET /identity/v3/role_assignments?group.id=01363f1eb6934ebf88c6f0b95ff1bb3c&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:43.185205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1af92-9d44-4293-8335-b13e06be1fbf None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:43.249026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:43.296522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f681d-dc82-4e06-abe2-b873a4e2fe8d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:43.304194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1af92-9d44-4293-8335-b13e06be1fbf None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:43.337730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2578fe7-da69-41a9-a4dd-95eed39446a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:43.338898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2578fe7-da69-41a9-a4dd-95eed39446a0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.339459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2578fe7-da69-41a9-a4dd-95eed39446a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.339917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2578fe7-da69-41a9-a4dd-95eed39446a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.340843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2578fe7-da69-41a9-a4dd-95eed39446a0 None admin] RBAC: Adding query filter params (role.id=053f81c391444db1b6409c4eb953dc35, user.id=04ab4ca23e7a4bfdb4bb717732648e44, scope.system=all) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:43.341349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2578fe7-da69-41a9-a4dd-95eed39446a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:43.343001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2578fe7-da69-41a9-a4dd-95eed39446a0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:43.356246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 189/375] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 10 09:50:43 2022] GET /identity/v3/role_assignments?user.id=04ab4ca23e7a4bfdb4bb717732648e44&scope.system=all&role.id=053f81c391444db1b6409c4eb953dc35 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:43.358498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2578fe7-da69-41a9-a4dd-95eed39446a0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:43.364312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d179f39-92f1-4453-a55f-9c6ac0abdaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:43.365185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d179f39-92f1-4453-a55f-9c6ac0abdaac None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.365543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d179f39-92f1-4453-a55f-9c6ac0abdaac None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.365887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d179f39-92f1-4453-a55f-9c6ac0abdaac None admin] PATH_INFO: `/v3/groups/590a9b9a5cea4037b31e3cd2e37d7f1c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.374275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d179f39-92f1-4453-a55f-9c6ac0abdaac None admin] RBAC: Authorizing `identity:get_group(group_id=590a9b9a5cea4037b31e3cd2e37d7f1c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:43.376741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d179f39-92f1-4453-a55f-9c6ac0abdaac None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:43.379784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 187/376] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:43 2022] GET /identity/v3/groups/590a9b9a5cea4037b31e3cd2e37d7f1c => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:43.381648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b53b-f823-4e4f-99d5-81be5a1ea492 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.382160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b53b-f823-4e4f-99d5-81be5a1ea492 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.382617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b53b-f823-4e4f-99d5-81be5a1ea492 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.385406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 188/377] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:43.386736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790b53b-f823-4e4f-99d5-81be5a1ea492 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:43.418233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:43.510292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2578fe7-da69-41a9-a4dd-95eed39446a0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:43.566895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20959f6f-e8de-4505-882d-9c1318863727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:43.569013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20959f6f-e8de-4505-882d-9c1318863727 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.570215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20959f6f-e8de-4505-882d-9c1318863727 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.571074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20959f6f-e8de-4505-882d-9c1318863727 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.572814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20959f6f-e8de-4505-882d-9c1318863727 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:43.574268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20959f6f-e8de-4505-882d-9c1318863727 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:43.576013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790b53b-f823-4e4f-99d5-81be5a1ea492 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:43.576141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 190/378] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:43 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:43.577257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cea6bf-76c1-4055-a185-bd0410ba97a9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.577545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cea6bf-76c1-4055-a185-bd0410ba97a9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.577821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cea6bf-76c1-4055-a185-bd0410ba97a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.592496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:43.600535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c3cea6bf-76c1-4055-a185-bd0410ba97a9 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:43.601769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c3cea6bf-76c1-4055-a185-bd0410ba97a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:43.622256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bad97-54c0-418b-affa-6d7c05f10995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:43.623157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bad97-54c0-418b-affa-6d7c05f10995 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.623434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bad97-54c0-418b-affa-6d7c05f10995 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.625769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bad97-54c0-418b-affa-6d7c05f10995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.625769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bad97-54c0-418b-affa-6d7c05f10995 None admin] RBAC: Adding query filter params (group.id=590a9b9a5cea4037b31e3cd2e37d7f1c, role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=9cee49555c8143ed868a4be75b65ed07) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:43.625962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bad97-54c0-418b-affa-6d7c05f10995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:43.627437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bad97-54c0-418b-affa-6d7c05f10995 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:43.638964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 189/379] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:43 2022] GET /identity/v3/role_assignments?group.id=590a9b9a5cea4037b31e3cd2e37d7f1c&scope.project.id=9cee49555c8143ed868a4be75b65ed07&role.id=8417343e9ef64991bcf51bb9692da102 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:43.640271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bad97-54c0-418b-affa-6d7c05f10995 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:43.696612 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 191/380] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:43.726882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cea6bf-76c1-4055-a185-bd0410ba97a9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:43.775875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bad97-54c0-418b-affa-6d7c05f10995 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:43.785652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:43.837450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555f9a3-bdb3-4bea-b196-ab4c53767018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:43.838624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555f9a3-bdb3-4bea-b196-ab4c53767018 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.840656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555f9a3-bdb3-4bea-b196-ab4c53767018 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.841132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555f9a3-bdb3-4bea-b196-ab4c53767018 None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.843709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555f9a3-bdb3-4bea-b196-ab4c53767018 None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:43.846749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555f9a3-bdb3-4bea-b196-ab4c53767018 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:43.850540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 190/381] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:43 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:43.856381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555f9a3-bdb3-4bea-b196-ab4c53767018 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:43.907518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cea6bf-76c1-4055-a185-bd0410ba97a9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:43.957568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1b086-353f-4554-9f62-701d3aed9bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:43.958501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1b086-353f-4554-9f62-701d3aed9bcb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:43.958945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1b086-353f-4554-9f62-701d3aed9bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:43.959402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1b086-353f-4554-9f62-701d3aed9bcb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:43.965098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:43.978482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1b086-353f-4554-9f62-701d3aed9bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-8ed1b086-353f-4554-9f62-701d3aed9bcb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:43.985109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:43.986015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:43.986805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 192/382] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:43.994109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1b086-353f-4554-9f62-701d3aed9bcb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:44.056562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555f9a3-bdb3-4bea-b196-ab4c53767018 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:44.104969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d2912-ad0e-4b8a-b903-501fb8c12b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:44.105530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d2912-ad0e-4b8a-b903-501fb8c12b92 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:44.105809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d2912-ad0e-4b8a-b903-501fb8c12b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:44.106077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d2912-ad0e-4b8a-b903-501fb8c12b92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:44.107254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d2912-ad0e-4b8a-b903-501fb8c12b92 None admin] RBAC: Adding query filter params (group.id=01363f1eb6934ebf88c6f0b95ff1bb3c, role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=d254037584f44b54b09c2b3e76807918) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:44.107577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d2912-ad0e-4b8a-b903-501fb8c12b92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:44.109358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d2912-ad0e-4b8a-b903-501fb8c12b92 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:44.110468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:44.118279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 191/383] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:43 2022] GET /identity/v3/role_assignments?group.id=01363f1eb6934ebf88c6f0b95ff1bb3c&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 260 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:44.120578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7584b-e7fd-4fef-9c37-759c6e8cf2a3 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:44.120869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7584b-e7fd-4fef-9c37-759c6e8cf2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:44.121494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7584b-e7fd-4fef-9c37-759c6e8cf2a3 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:44.123159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 192/384] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:44.123498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1b086-353f-4554-9f62-701d3aed9bcb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:44.129721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04babc3-5b50-46f5-82c2-393f92900fd2 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:44.130033 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04babc3-5b50-46f5-82c2-393f92900fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:44.130279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04babc3-5b50-46f5-82c2-393f92900fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:44.153526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-b04babc3-5b50-46f5-82c2-393f92900fd2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:44.154746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b04babc3-5b50-46f5-82c2-393f92900fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:44.175084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915c23f-9613-405e-8aa5-b792357057f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:44.175753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915c23f-9613-405e-8aa5-b792357057f8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:44.176131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915c23f-9613-405e-8aa5-b792357057f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:44.176413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915c23f-9613-405e-8aa5-b792357057f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:44.177104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915c23f-9613-405e-8aa5-b792357057f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:44.177399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915c23f-9613-405e-8aa5-b792357057f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:44.178857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915c23f-9613-405e-8aa5-b792357057f8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:44.189365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 193/385] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:44.195175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915c23f-9613-405e-8aa5-b792357057f8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:44.237975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 193/386] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:44.265125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04babc3-5b50-46f5-82c2-393f92900fd2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:44.314980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:44.327014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915c23f-9613-405e-8aa5-b792357057f8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:44.369181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ef2a2-2520-46e7-b094-09bc55e031fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:44.371013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ef2a2-2520-46e7-b094-09bc55e031fc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:44.371013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ef2a2-2520-46e7-b094-09bc55e031fc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:44.371013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ef2a2-2520-46e7-b094-09bc55e031fc None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:44.371622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ef2a2-2520-46e7-b094-09bc55e031fc None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:44.373082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ef2a2-2520-46e7-b094-09bc55e031fc None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:44.380157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 194/387] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:44 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:44.382976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04babc3-5b50-46f5-82c2-393f92900fd2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:44.383120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ef2a2-2520-46e7-b094-09bc55e031fc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:44.426981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:44.438987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1f279-874e-4745-b1de-a8a5e56ffe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:44.438987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1f279-874e-4745-b1de-a8a5e56ffe00 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:44.438987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1f279-874e-4745-b1de-a8a5e56ffe00 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:44.438987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1f279-874e-4745-b1de-a8a5e56ffe00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:44.458979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1f279-874e-4745-b1de-a8a5e56ffe00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-e8d1f279-874e-4745-b1de-a8a5e56ffe00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:44.462977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:44.463884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:44.464647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:44.474978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 194/388] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:44.474978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1f279-874e-4745-b1de-a8a5e56ffe00 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:44.565922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ef2a2-2520-46e7-b094-09bc55e031fc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:44.661399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:44.669623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1f279-874e-4745-b1de-a8a5e56ffe00 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:44.684349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b70f63-5931-4d1e-8bc5-f115aaed943f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:44.685885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70f63-5931-4d1e-8bc5-f115aaed943f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:44.686357 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70f63-5931-4d1e-8bc5-f115aaed943f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:44.686827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70f63-5931-4d1e-8bc5-f115aaed943f None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:44.689120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b70f63-5931-4d1e-8bc5-f115aaed943f None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:44.690795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b70f63-5931-4d1e-8bc5-f115aaed943f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:44.693420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 195/389] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:44 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:44.699978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b70f63-5931-4d1e-8bc5-f115aaed943f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:44.770216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa492fb1-4cc6-47aa-bc16-e6a32f50a9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:44.772662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492fb1-4cc6-47aa-bc16-e6a32f50a9cc None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:44.773137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492fb1-4cc6-47aa-bc16-e6a32f50a9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:44.774705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492fb1-4cc6-47aa-bc16-e6a32f50a9cc None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:44.777059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa492fb1-4cc6-47aa-bc16-e6a32f50a9cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:44.778971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa492fb1-4cc6-47aa-bc16-e6a32f50a9cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:44.782140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa492fb1-4cc6-47aa-bc16-e6a32f50a9cc None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:44.816843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 195/390] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:44.818661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa492fb1-4cc6-47aa-bc16-e6a32f50a9cc None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:44.985625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:45.016449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa492fb1-4cc6-47aa-bc16-e6a32f50a9cc None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:45.032974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b70f63-5931-4d1e-8bc5-f115aaed943f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:45.075299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602c7d9-ab92-4dbd-b1cd-b2a05a6f20ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:45.076290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c7d9-ab92-4dbd-b1cd-b2a05a6f20ff None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:45.078757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c7d9-ab92-4dbd-b1cd-b2a05a6f20ff None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:45.080301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c7d9-ab92-4dbd-b1cd-b2a05a6f20ff None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:45.084792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602c7d9-ab92-4dbd-b1cd-b2a05a6f20ff None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:45.088593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602c7d9-ab92-4dbd-b1cd-b2a05a6f20ff None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:45.093886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 196/391] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:44 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:45.096888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602c7d9-ab92-4dbd-b1cd-b2a05a6f20ff None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:45.123662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe55874-83b7-4be6-a22b-ab966a88ead4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:45.127519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe55874-83b7-4be6-a22b-ab966a88ead4 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:45.128286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe55874-83b7-4be6-a22b-ab966a88ead4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:45.129601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe55874-83b7-4be6-a22b-ab966a88ead4 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/groups/01363f1eb6934ebf88c6f0b95ff1bb3c/roles/02c7974bceec49ee8895dabf30cc8fff` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:45.137068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe55874-83b7-4be6-a22b-ab966a88ead4 None admin] RBAC: Authorizing `identity:create_grant(project_id=02ab39fdbee942d8bd432a27ca1d13c5, group_id=01363f1eb6934ebf88c6f0b95ff1bb3c, role_id=02c7974bceec49ee8895dabf30cc8fff)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:45.138530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe55874-83b7-4be6-a22b-ab966a88ead4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:45.162499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 196/392] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 10 09:50:44 2022] PUT /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/groups/01363f1eb6934ebf88c6f0b95ff1bb3c/roles/02c7974bceec49ee8895dabf30cc8fff => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:45.163141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:45.243231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602c7d9-ab92-4dbd-b1cd-b2a05a6f20ff None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:45.279264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad610e4a-ad46-4818-b17e-ae5b5bfb8a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:45.279958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad610e4a-ad46-4818-b17e-ae5b5bfb8a05 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:45.280239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad610e4a-ad46-4818-b17e-ae5b5bfb8a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:45.280508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad610e4a-ad46-4818-b17e-ae5b5bfb8a05 None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:45.281664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad610e4a-ad46-4818-b17e-ae5b5bfb8a05 None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:45.283134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad610e4a-ad46-4818-b17e-ae5b5bfb8a05 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:45.285072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 197/393] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:45 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:45.290857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe55874-83b7-4be6-a22b-ab966a88ead4 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:45.423674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe55874-83b7-4be6-a22b-ab966a88ead4 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:45.468823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2c31b-9783-4474-bb01-133da30b8d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:45.469882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2c31b-9783-4474-bb01-133da30b8d5e None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:45.470370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2c31b-9783-4474-bb01-133da30b8d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:45.470842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2c31b-9783-4474-bb01-133da30b8d5e None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/groups/01363f1eb6934ebf88c6f0b95ff1bb3c/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:45.472991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2c31b-9783-4474-bb01-133da30b8d5e None admin] RBAC: Authorizing `identity:create_grant(project_id=02ab39fdbee942d8bd432a27ca1d13c5, group_id=01363f1eb6934ebf88c6f0b95ff1bb3c, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:45.474646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2c31b-9783-4474-bb01-133da30b8d5e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:45.496820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 197/394] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 10 09:50:45 2022] PUT /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5/groups/01363f1eb6934ebf88c6f0b95ff1bb3c/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:45.919992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e2fa1-86f6-4a70-b1cd-2610870d5b37 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:45.920362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e2fa1-86f6-4a70-b1cd-2610870d5b37 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:45.920564 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e2fa1-86f6-4a70-b1cd-2610870d5b37 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:45.922137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 198/395] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:45.927814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c43ff-2ee7-4841-a243-36188c624dc5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:45.928100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c43ff-2ee7-4841-a243-36188c624dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:45.928380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c43ff-2ee7-4841-a243-36188c624dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:45.951053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3b9c43ff-2ee7-4841-a243-36188c624dc5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:45.952244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3b9c43ff-2ee7-4841-a243-36188c624dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:45.955378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:46.031071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 198/396] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:46.049903 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e2fa1-86f6-4a70-b1cd-2610870d5b37 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:46.165906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:46.174613 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e2fa1-86f6-4a70-b1cd-2610870d5b37 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:46.213169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef994c5-b348-4e53-ac3d-eb8948422c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:46.213863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef994c5-b348-4e53-ac3d-eb8948422c53 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:46.214146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef994c5-b348-4e53-ac3d-eb8948422c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:46.214419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef994c5-b348-4e53-ac3d-eb8948422c53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:46.227681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef994c5-b348-4e53-ac3d-eb8948422c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-3ef994c5-b348-4e53-ac3d-eb8948422c53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:46.232573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:46.233427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:46.234210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:46.237128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 199/397] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:46.240086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c43ff-2ee7-4841-a243-36188c624dc5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:46.365182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c43ff-2ee7-4841-a243-36188c624dc5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:46.420109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb37c30-7c1c-4da0-898a-5194fb19a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:46.421115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb37c30-7c1c-4da0-898a-5194fb19a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:46.421739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb37c30-7c1c-4da0-898a-5194fb19a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:46.422201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb37c30-7c1c-4da0-898a-5194fb19a2a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:46.423155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb37c30-7c1c-4da0-898a-5194fb19a2a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:46.423775 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb37c30-7c1c-4da0-898a-5194fb19a2a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:46.425383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb37c30-7c1c-4da0-898a-5194fb19a2a1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:46.438133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 199/398] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:46.443993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef994c5-b348-4e53-ac3d-eb8948422c53 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:46.587748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef994c5-b348-4e53-ac3d-eb8948422c53 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:46.593596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4bc92-59f4-4f75-b62c-23be4f86724b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:46.593880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4bc92-59f4-4f75-b62c-23be4f86724b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:46.594155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4bc92-59f4-4f75-b62c-23be4f86724b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:46.595527 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 200/399] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:46.609074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b56b-e0f6-4f89-909d-acf09c538d26 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:46.609379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b56b-e0f6-4f89-909d-acf09c538d26 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:46.609623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b56b-e0f6-4f89-909d-acf09c538d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:46.620713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:46.639596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-eaf3b56b-e0f6-4f89-909d-acf09c538d26 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:46.640954 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-eaf3b56b-e0f6-4f89-909d-acf09c538d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:46.655527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce82c9-0b27-41db-83d1-784d399b4c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:46.656540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce82c9-0b27-41db-83d1-784d399b4c29 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:46.657010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce82c9-0b27-41db-83d1-784d399b4c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:46.657479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce82c9-0b27-41db-83d1-784d399b4c29 None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:46.658833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce82c9-0b27-41db-83d1-784d399b4c29 None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:46.660570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce82c9-0b27-41db-83d1-784d399b4c29 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:46.662310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 200/400] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:46 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:46.668734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce82c9-0b27-41db-83d1-784d399b4c29 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:46.723388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:46.776630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 201/401] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:46.798149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8edff1-b291-41e3-958b-65b8877d1f47 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:46.798435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8edff1-b291-41e3-958b-65b8877d1f47 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:46.798678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8edff1-b291-41e3-958b-65b8877d1f47 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:46.800264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 202/402] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:46.806920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc077b-cd67-4115-bc05-3f74901f828e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:46.807216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc077b-cd67-4115-bc05-3f74901f828e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:46.807486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc077b-cd67-4115-bc05-3f74901f828e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:46.810667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce82c9-0b27-41db-83d1-784d399b4c29 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:46.827065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-35dc077b-cd67-4115-bc05-3f74901f828e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:46.828899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-35dc077b-cd67-4115-bc05-3f74901f828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:46.851707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4be02-6299-4798-8bc0-d3c935e14499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:46.852361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4be02-6299-4798-8bc0-d3c935e14499 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:46.852641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4be02-6299-4798-8bc0-d3c935e14499 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:46.852915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4be02-6299-4798-8bc0-d3c935e14499 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:46.854280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4be02-6299-4798-8bc0-d3c935e14499 None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:46.855799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4be02-6299-4798-8bc0-d3c935e14499 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:46.857653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 201/403] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:46 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:46.863695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4be02-6299-4798-8bc0-d3c935e14499 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:46.910403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 203/404] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:46.911211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc077b-cd67-4115-bc05-3f74901f828e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:46.988149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:46.991232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:47.004200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4be02-6299-4798-8bc0-d3c935e14499 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:47.032896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc077b-cd67-4115-bc05-3f74901f828e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:47.049730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b287e-41c3-41ff-b2d0-6869aa016c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:47.050603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b287e-41c3-41ff-b2d0-6869aa016c7b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:47.051283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b287e-41c3-41ff-b2d0-6869aa016c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:47.051557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b287e-41c3-41ff-b2d0-6869aa016c7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:47.065814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b287e-41c3-41ff-b2d0-6869aa016c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:47.068629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6cb26-b76b-478d-9e75-5ffa9a0871de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:47.069389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6cb26-b76b-478d-9e75-5ffa9a0871de None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:47.069769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6cb26-b76b-478d-9e75-5ffa9a0871de None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:47.070186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6cb26-b76b-478d-9e75-5ffa9a0871de None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918/groups/01363f1eb6934ebf88c6f0b95ff1bb3c/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-a87b287e-41c3-41ff-b2d0-6869aa016c7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:47.070865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:47.072788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 202/405] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:47.074572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b287e-41c3-41ff-b2d0-6869aa016c7b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:47.076074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6cb26-b76b-478d-9e75-5ffa9a0871de None admin] RBAC: Authorizing `identity:create_grant(project_id=d254037584f44b54b09c2b3e76807918, group_id=01363f1eb6934ebf88c6f0b95ff1bb3c, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:47.076074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6cb26-b76b-478d-9e75-5ffa9a0871de None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:47.099145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 204/406] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 10 09:50:46 2022] PUT /identity/v3/projects/d254037584f44b54b09c2b3e76807918/groups/01363f1eb6934ebf88c6f0b95ff1bb3c/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:47.100346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6cb26-b76b-478d-9e75-5ffa9a0871de None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:47.203952 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b287e-41c3-41ff-b2d0-6869aa016c7b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:47.244135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ea54d-8bf2-4877-8e09-7f8f17c18d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:47.244688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ea54d-8bf2-4877-8e09-7f8f17c18d44 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:47.244963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ea54d-8bf2-4877-8e09-7f8f17c18d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:47.245237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ea54d-8bf2-4877-8e09-7f8f17c18d44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:47.256689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ea54d-8bf2-4877-8e09-7f8f17c18d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-959ea54d-8bf2-4877-8e09-7f8f17c18d44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:47.262057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:47.264095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 203/407] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:47.271503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ea54d-8bf2-4877-8e09-7f8f17c18d44 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:47.287783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:47.329609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6cb26-b76b-478d-9e75-5ffa9a0871de None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:47.376834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:47.393346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8edf6-cd0a-42f9-a211-077969a29330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:47.394019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8edf6-cd0a-42f9-a211-077969a29330 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:47.394305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8edf6-cd0a-42f9-a211-077969a29330 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:47.394581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8edf6-cd0a-42f9-a211-077969a29330 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:47.395312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8edf6-cd0a-42f9-a211-077969a29330 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:47.395618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ea54d-8bf2-4877-8e09-7f8f17c18d44 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:47.395723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8edf6-cd0a-42f9-a211-077969a29330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:47.397086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8edf6-cd0a-42f9-a211-077969a29330 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:47.411312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 205/408] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:47.415115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8edf6-cd0a-42f9-a211-077969a29330 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:47.443994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620d58b-89a7-42d2-abf7-f207885a6463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:47.444943 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620d58b-89a7-42d2-abf7-f207885a6463 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:47.445397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620d58b-89a7-42d2-abf7-f207885a6463 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:47.445788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620d58b-89a7-42d2-abf7-f207885a6463 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:47.446463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620d58b-89a7-42d2-abf7-f207885a6463 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:47.446923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620d58b-89a7-42d2-abf7-f207885a6463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:47.448915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620d58b-89a7-42d2-abf7-f207885a6463 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:47.459300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 204/409] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:47.465214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620d58b-89a7-42d2-abf7-f207885a6463 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:47.533756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8edf6-cd0a-42f9-a211-077969a29330 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:47.577850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5c040-aa01-4fdd-879b-37b011dff019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:47.581064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5c040-aa01-4fdd-879b-37b011dff019 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:47.581342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5c040-aa01-4fdd-879b-37b011dff019 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:47.581613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5c040-aa01-4fdd-879b-37b011dff019 None admin] PATH_INFO: `/v3/groups/590a9b9a5cea4037b31e3cd2e37d7f1c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:47.582791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5c040-aa01-4fdd-879b-37b011dff019 None admin] RBAC: Authorizing `identity:get_group(group_id=590a9b9a5cea4037b31e3cd2e37d7f1c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:47.585281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5c040-aa01-4fdd-879b-37b011dff019 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:47.587588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 206/410] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:47 2022] GET /identity/v3/groups/590a9b9a5cea4037b31e3cd2e37d7f1c => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:47.593503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5c040-aa01-4fdd-879b-37b011dff019 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:47.606617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:47.654795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620d58b-89a7-42d2-abf7-f207885a6463 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:47.698918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:47.749019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5c040-aa01-4fdd-879b-37b011dff019 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:47.763672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae01101-07f3-4ea7-9bc2-10491006535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:47.764328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae01101-07f3-4ea7-9bc2-10491006535b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:47.764611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae01101-07f3-4ea7-9bc2-10491006535b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:47.764889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae01101-07f3-4ea7-9bc2-10491006535b None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:47.771630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae01101-07f3-4ea7-9bc2-10491006535b None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:47.773094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae01101-07f3-4ea7-9bc2-10491006535b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:47.775305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 205/411] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:47 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:47.780619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae01101-07f3-4ea7-9bc2-10491006535b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:47.797747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffe95d-ea71-4c7b-9124-10c37f6b08a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:47.798379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffe95d-ea71-4c7b-9124-10c37f6b08a8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:47.798656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffe95d-ea71-4c7b-9124-10c37f6b08a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:47.798928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffe95d-ea71-4c7b-9124-10c37f6b08a8 None admin] PATH_INFO: `/v3/projects/9cee49555c8143ed868a4be75b65ed07` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:47.801145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ffe95d-ea71-4c7b-9124-10c37f6b08a8 None admin] RBAC: Authorizing `identity:get_project(project_id=9cee49555c8143ed868a4be75b65ed07)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:47.802570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ffe95d-ea71-4c7b-9124-10c37f6b08a8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:47.805458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 207/412] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:47 2022] GET /identity/v3/projects/9cee49555c8143ed868a4be75b65ed07 => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:47.815868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffe95d-ea71-4c7b-9124-10c37f6b08a8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:47.897136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae01101-07f3-4ea7-9bc2-10491006535b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:47.952474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dcd43-39fa-45bf-a90e-3110ae85310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:47.953111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dcd43-39fa-45bf-a90e-3110ae85310d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:47.953388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dcd43-39fa-45bf-a90e-3110ae85310d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:47.953660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dcd43-39fa-45bf-a90e-3110ae85310d None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:47.955903 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dcd43-39fa-45bf-a90e-3110ae85310d None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:47.957352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dcd43-39fa-45bf-a90e-3110ae85310d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:47.963568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 206/413] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:47 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:47.968184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dcd43-39fa-45bf-a90e-3110ae85310d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:47.994583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:48.022812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffe95d-ea71-4c7b-9124-10c37f6b08a8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:48.049613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:48.076613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1455b6-d067-4862-9253-4d56ea3069f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:48.077526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1455b6-d067-4862-9253-4d56ea3069f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.077974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1455b6-d067-4862-9253-4d56ea3069f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.078408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1455b6-d067-4862-9253-4d56ea3069f8 None admin] PATH_INFO: `/v3/projects/9cee49555c8143ed868a4be75b65ed07/groups/590a9b9a5cea4037b31e3cd2e37d7f1c/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.080566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1455b6-d067-4862-9253-4d56ea3069f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cee49555c8143ed868a4be75b65ed07, group_id=590a9b9a5cea4037b31e3cd2e37d7f1c, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:48.082761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1455b6-d067-4862-9253-4d56ea3069f8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:48.107467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 208/414] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 10 09:50:47 2022] PUT /identity/v3/projects/9cee49555c8143ed868a4be75b65ed07/groups/590a9b9a5cea4037b31e3cd2e37d7f1c/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:48.108651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325125e-e6bd-4941-879f-b28544a53be7 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.108932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325125e-e6bd-4941-879f-b28544a53be7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.110172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325125e-e6bd-4941-879f-b28544a53be7 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.119672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 209/415] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:48 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:48.122524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dcd43-39fa-45bf-a90e-3110ae85310d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:48.126694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6e5f0-50c0-4248-8362-abeede6e2dd3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.127060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6e5f0-50c0-4248-8362-abeede6e2dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.127433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6e5f0-50c0-4248-8362-abeede6e2dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.161027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-05e6e5f0-50c0-4248-8362-abeede6e2dd3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:48.164898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-05e6e5f0-50c0-4248-8362-abeede6e2dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:48.203635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de8974-b84e-40c5-aa1b-3d612dbac7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:48.204560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de8974-b84e-40c5-aa1b-3d612dbac7e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.204837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de8974-b84e-40c5-aa1b-3d612dbac7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.205106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de8974-b84e-40c5-aa1b-3d612dbac7e7 None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918/groups/01363f1eb6934ebf88c6f0b95ff1bb3c/roles/02c7974bceec49ee8895dabf30cc8fff` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.210219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de8974-b84e-40c5-aa1b-3d612dbac7e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d254037584f44b54b09c2b3e76807918, group_id=01363f1eb6934ebf88c6f0b95ff1bb3c, role_id=02c7974bceec49ee8895dabf30cc8fff)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:48.213234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de8974-b84e-40c5-aa1b-3d612dbac7e7 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:48.240874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 207/416] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 10 09:50:47 2022] PUT /identity/v3/projects/d254037584f44b54b09c2b3e76807918/groups/01363f1eb6934ebf88c6f0b95ff1bb3c/roles/02c7974bceec49ee8895dabf30cc8fff => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:50:48.279737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 210/417] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:48.303293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de8974-b84e-40c5-aa1b-3d612dbac7e7 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:48.386600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:48.422186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de8974-b84e-40c5-aa1b-3d612dbac7e7 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:48.458606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e13f4-ca69-4144-9be1-5babf50cf036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:48.459266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e13f4-ca69-4144-9be1-5babf50cf036 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.459523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e13f4-ca69-4144-9be1-5babf50cf036 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.459791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e13f4-ca69-4144-9be1-5babf50cf036 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.470456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e13f4-ca69-4144-9be1-5babf50cf036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-375e13f4-ca69-4144-9be1-5babf50cf036 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:48.475700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:48.477092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:48.477964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 208/418] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:48.481592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441850ab-b9e7-420f-8ded-b601eb940f88 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.482119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441850ab-b9e7-420f-8ded-b601eb940f88 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.482119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441850ab-b9e7-420f-8ded-b601eb940f88 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.483801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e13f4-ca69-4144-9be1-5babf50cf036 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:48.487421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 211/419] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:48.489079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c353b-de0d-4295-a71b-7159ba613f91 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.489364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c353b-de0d-4295-a71b-7159ba613f91 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.489645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c353b-de0d-4295-a71b-7159ba613f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.512358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-683c353b-de0d-4295-a71b-7159ba613f91 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:48.513652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-683c353b-de0d-4295-a71b-7159ba613f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:48.608538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 212/420] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:48.633806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e13f4-ca69-4144-9be1-5babf50cf036 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:48.668098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:48.678344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4951d-e83c-4812-bc1e-4013c1c8348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:48.678632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c353b-de0d-4295-a71b-7159ba613f91 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:48.678987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4951d-e83c-4812-bc1e-4013c1c8348f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.679338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4951d-e83c-4812-bc1e-4013c1c8348f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.679610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4951d-e83c-4812-bc1e-4013c1c8348f None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.680848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4951d-e83c-4812-bc1e-4013c1c8348f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:48.681275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4951d-e83c-4812-bc1e-4013c1c8348f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:48.682655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4951d-e83c-4812-bc1e-4013c1c8348f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:48.692454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 209/421] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:48.703956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4951d-e83c-4812-bc1e-4013c1c8348f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:48.734791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:48.841216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c353b-de0d-4295-a71b-7159ba613f91 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:48.894013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4951d-e83c-4812-bc1e-4013c1c8348f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:48.924221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c18240-2639-4cc1-a67b-f14920e6b448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:48.924873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c18240-2639-4cc1-a67b-f14920e6b448 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.925153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c18240-2639-4cc1-a67b-f14920e6b448 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.925425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c18240-2639-4cc1-a67b-f14920e6b448 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.933195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00738-095a-4def-8098-21ac2d5321b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:48.933796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00738-095a-4def-8098-21ac2d5321b6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:48.934069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00738-095a-4def-8098-21ac2d5321b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:48.934334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00738-095a-4def-8098-21ac2d5321b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:48.950851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c18240-2639-4cc1-a67b-f14920e6b448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-a0c18240-2639-4cc1-a67b-f14920e6b448 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:48.955647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:48.956549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:48.957316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 213/422] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:48.967930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c18240-2639-4cc1-a67b-f14920e6b448 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:48.999379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:49.008138 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00738-095a-4def-8098-21ac2d5321b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:49.009676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00738-095a-4def-8098-21ac2d5321b6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:49.113385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 210/423] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:49.120009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00738-095a-4def-8098-21ac2d5321b6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:49.219448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:49.219929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:49.227683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00738-095a-4def-8098-21ac2d5321b6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:49.229066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c18240-2639-4cc1-a67b-f14920e6b448 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:49.265540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016cafa-d11e-4cce-8d9f-ea606e66d16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:49.266269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016cafa-d11e-4cce-8d9f-ea606e66d16a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:49.266618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016cafa-d11e-4cce-8d9f-ea606e66d16a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:49.266980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016cafa-d11e-4cce-8d9f-ea606e66d16a None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:49.268306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016cafa-d11e-4cce-8d9f-ea606e66d16a None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:49.269886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016cafa-d11e-4cce-8d9f-ea606e66d16a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:49.272175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 211/424] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:49 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:49.278638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016cafa-d11e-4cce-8d9f-ea606e66d16a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:49.352302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c056bd7-9951-4717-bb0a-75f9cab51227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:49.352924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c056bd7-9951-4717-bb0a-75f9cab51227 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:49.353201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c056bd7-9951-4717-bb0a-75f9cab51227 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:49.353472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c056bd7-9951-4717-bb0a-75f9cab51227 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:49.354151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c056bd7-9951-4717-bb0a-75f9cab51227 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:49.354450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c056bd7-9951-4717-bb0a-75f9cab51227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:49.355848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c056bd7-9951-4717-bb0a-75f9cab51227 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:49.380677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016cafa-d11e-4cce-8d9f-ea606e66d16a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:49.389131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 214/425] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:49.401766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c056bd7-9951-4717-bb0a-75f9cab51227 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:49.425070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d306610-c9cd-45b6-b8a7-05b71ba840dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:49.425710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d306610-c9cd-45b6-b8a7-05b71ba840dc None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:49.425988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d306610-c9cd-45b6-b8a7-05b71ba840dc None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:49.426304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d306610-c9cd-45b6-b8a7-05b71ba840dc None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:49.429291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d306610-c9cd-45b6-b8a7-05b71ba840dc None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:49.430764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d306610-c9cd-45b6-b8a7-05b71ba840dc None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:49.439117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 212/426] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:49 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:49.453956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d306610-c9cd-45b6-b8a7-05b71ba840dc None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:49.511398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:49.574082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c056bd7-9951-4717-bb0a-75f9cab51227 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:49.602406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d306610-c9cd-45b6-b8a7-05b71ba840dc None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:49.610628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c22b-a313-4acf-b976-7d54b7ace59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:49.611664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c22b-a313-4acf-b976-7d54b7ace59b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:49.612162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c22b-a313-4acf-b976-7d54b7ace59b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:49.612633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c22b-a313-4acf-b976-7d54b7ace59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:49.644940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddb545-b9ca-4719-9842-4e73879bbca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:49.645793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddb545-b9ca-4719-9842-4e73879bbca5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:49.646071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddb545-b9ca-4719-9842-4e73879bbca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:49.646343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddb545-b9ca-4719-9842-4e73879bbca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:49.647128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddb545-b9ca-4719-9842-4e73879bbca5 None admin] RBAC: Adding query filter params (group.id=01363f1eb6934ebf88c6f0b95ff1bb3c, role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:49.647434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddb545-b9ca-4719-9842-4e73879bbca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:49.648268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:49.649554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddb545-b9ca-4719-9842-4e73879bbca5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:49.659040 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 213/427] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:49 2022] GET /identity/v3/role_assignments?group.id=01363f1eb6934ebf88c6f0b95ff1bb3c&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:49.660728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c8680-e9e7-49ff-b0af-4220b683c0b2 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:49.661275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c8680-e9e7-49ff-b0af-4220b683c0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:49.661805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c8680-e9e7-49ff-b0af-4220b683c0b2 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:49.673890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 214/428] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:49.680355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109b753-0865-4f4a-bd39-6d71c6ed52d1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:49.681010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109b753-0865-4f4a-bd39-6d71c6ed52d1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:49.681482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109b753-0865-4f4a-bd39-6d71c6ed52d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:49.730434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-4109b753-0865-4f4a-bd39-6d71c6ed52d1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:49.732821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4109b753-0865-4f4a-bd39-6d71c6ed52d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:49.756381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4c22b-a313-4acf-b976-7d54b7ace59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:49.759393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4c22b-a313-4acf-b976-7d54b7ace59b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:49.879029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 215/429] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:49.901715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109b753-0865-4f4a-bd39-6d71c6ed52d1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:49.927487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 215/430] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:49.934521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c22b-a313-4acf-b976-7d54b7ace59b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:49.936791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:50.000337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:50.029401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109b753-0865-4f4a-bd39-6d71c6ed52d1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:50.042874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c22b-a313-4acf-b976-7d54b7ace59b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:50.073881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920f53a-41ce-43a0-a0f2-1b0e71796d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:50.074886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920f53a-41ce-43a0-a0f2-1b0e71796d09 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.075409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920f53a-41ce-43a0-a0f2-1b0e71796d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.075879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920f53a-41ce-43a0-a0f2-1b0e71796d09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.081647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394b20d-e879-4641-9e02-48cd2be4f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:50.082291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394b20d-e879-4641-9e02-48cd2be4f3dc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.082564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394b20d-e879-4641-9e02-48cd2be4f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.082830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394b20d-e879-4641-9e02-48cd2be4f3dc None admin] PATH_INFO: `/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.085053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394b20d-e879-4641-9e02-48cd2be4f3dc None admin] RBAC: Authorizing `identity:get_project(project_id=02ab39fdbee942d8bd432a27ca1d13c5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:50.087908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394b20d-e879-4641-9e02-48cd2be4f3dc None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:50.088983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 216/431] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:49 2022] GET /identity/v3/projects/02ab39fdbee942d8bd432a27ca1d13c5 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:50.092170 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920f53a-41ce-43a0-a0f2-1b0e71796d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:50.094153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394b20d-e879-4641-9e02-48cd2be4f3dc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-9920f53a-41ce-43a0-a0f2-1b0e71796d09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:50.098036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:50.098917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 09:50:50.100556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:50.108394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 216/432] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:50.109909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920f53a-41ce-43a0-a0f2-1b0e71796d09 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:50.204894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394b20d-e879-4641-9e02-48cd2be4f3dc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:50.215328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:50.226352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920f53a-41ce-43a0-a0f2-1b0e71796d09 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:50.234229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:50.240134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039a6d7-c9d4-46b9-8b6c-9489e8fca948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:50.241171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039a6d7-c9d4-46b9-8b6c-9489e8fca948 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.241547 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039a6d7-c9d4-46b9-8b6c-9489e8fca948 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.242004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039a6d7-c9d4-46b9-8b6c-9489e8fca948 None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.243742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039a6d7-c9d4-46b9-8b6c-9489e8fca948 None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:50.245278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039a6d7-c9d4-46b9-8b6c-9489e8fca948 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:50.251866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 217/433] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:50 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:50.252828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039a6d7-c9d4-46b9-8b6c-9489e8fca948 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:50.271209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a746cf0-9fad-4f0c-aeb0-0e4b931c6d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:50.272049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a746cf0-9fad-4f0c-aeb0-0e4b931c6d6b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.272498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a746cf0-9fad-4f0c-aeb0-0e4b931c6d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.272913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a746cf0-9fad-4f0c-aeb0-0e4b931c6d6b None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.273891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a746cf0-9fad-4f0c-aeb0-0e4b931c6d6b None admin] RBAC: Adding query filter params (name=member) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:50.274488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a746cf0-9fad-4f0c-aeb0-0e4b931c6d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:50.277233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a746cf0-9fad-4f0c-aeb0-0e4b931c6d6b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:50.291078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 217/434] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:50:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:50.293351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a746cf0-9fad-4f0c-aeb0-0e4b931c6d6b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:50.359572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039a6d7-c9d4-46b9-8b6c-9489e8fca948 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:50.393915 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfee34-44d1-4161-97b1-49de22ad9f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:50.394669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfee34-44d1-4161-97b1-49de22ad9f16 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.395092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfee34-44d1-4161-97b1-49de22ad9f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.395457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfee34-44d1-4161-97b1-49de22ad9f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.396272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfee34-44d1-4161-97b1-49de22ad9f16 None admin] RBAC: Adding query filter params (group.id=01363f1eb6934ebf88c6f0b95ff1bb3c, role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:50.396723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfee34-44d1-4161-97b1-49de22ad9f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:50.398262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfee34-44d1-4161-97b1-49de22ad9f16 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:50.405898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a746cf0-9fad-4f0c-aeb0-0e4b931c6d6b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:50.411280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 218/435] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:50 2022] GET /identity/v3/role_assignments?group.id=01363f1eb6934ebf88c6f0b95ff1bb3c&scope.project.id=02ab39fdbee942d8bd432a27ca1d13c5&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:50.446326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831581-20af-40bf-90d3-1c14902b4ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:50.447448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831581-20af-40bf-90d3-1c14902b4ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.447957 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831581-20af-40bf-90d3-1c14902b4ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.448425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831581-20af-40bf-90d3-1c14902b4ba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.467435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:50.500482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35479ab-a709-451f-91aa-d107b9b13e4b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.500762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35479ab-a709-451f-91aa-d107b9b13e4b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.501035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35479ab-a709-451f-91aa-d107b9b13e4b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.502502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 219/436] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:50.509995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd301700-49b8-4d58-8686-a687fe599f67 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.510567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd301700-49b8-4d58-8686-a687fe599f67 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.511470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd301700-49b8-4d58-8686-a687fe599f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.534300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831581-20af-40bf-90d3-1c14902b4ba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:50.535809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831581-20af-40bf-90d3-1c14902b4ba6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:50.536818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fd301700-49b8-4d58-8686-a687fe599f67 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:50.538439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fd301700-49b8-4d58-8686-a687fe599f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:50.616933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:50.632001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 220/437] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:50.633902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bebfa9-38e7-434a-adab-96146df96dca None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.634188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bebfa9-38e7-434a-adab-96146df96dca None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.634453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bebfa9-38e7-434a-adab-96146df96dca None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.636353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 221/438] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:50.641420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f161-3e8f-44a8-be5f-62d62ca94d85 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.641697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f161-3e8f-44a8-be5f-62d62ca94d85 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.641969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f161-3e8f-44a8-be5f-62d62ca94d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.663170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-19e9f161-3e8f-44a8-be5f-62d62ca94d85 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:50.665127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-19e9f161-3e8f-44a8-be5f-62d62ca94d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:50.665577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 218/439] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:50.666863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831581-20af-40bf-90d3-1c14902b4ba6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:50.716533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:50.739671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 222/440] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:50.747417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f161-3e8f-44a8-be5f-62d62ca94d85 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:50.775733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831581-20af-40bf-90d3-1c14902b4ba6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:50.819088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dd619-e840-47c7-9e2d-6696d084955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:50.819717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dd619-e840-47c7-9e2d-6696d084955c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.819999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dd619-e840-47c7-9e2d-6696d084955c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.820782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dd619-e840-47c7-9e2d-6696d084955c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.833801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148dd619-e840-47c7-9e2d-6696d084955c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-148dd619-e840-47c7-9e2d-6696d084955c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:50.840296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:50.841182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:50.841970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 219/441] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:50:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:50.842605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dd619-e840-47c7-9e2d-6696d084955c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:50.872340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f161-3e8f-44a8-be5f-62d62ca94d85 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:50.902512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:50.915301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b682cc-bb87-4bd5-8ff1-40cd457181cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:50.915940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b682cc-bb87-4bd5-8ff1-40cd457181cc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:50.916224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b682cc-bb87-4bd5-8ff1-40cd457181cc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:50.916494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b682cc-bb87-4bd5-8ff1-40cd457181cc None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:50.917743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b682cc-bb87-4bd5-8ff1-40cd457181cc None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:50.921416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b682cc-bb87-4bd5-8ff1-40cd457181cc None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:50.921416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 223/442] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:50 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:50.921611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b682cc-bb87-4bd5-8ff1-40cd457181cc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:50.969392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dd619-e840-47c7-9e2d-6696d084955c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:50.972421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:51.006349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8d92b-04d8-494e-bfab-13c54ba1d97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.006898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8d92b-04d8-494e-bfab-13c54ba1d97c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.007204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8d92b-04d8-494e-bfab-13c54ba1d97c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.007474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8d92b-04d8-494e-bfab-13c54ba1d97c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.017625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8d92b-04d8-494e-bfab-13c54ba1d97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-2ba8d92b-04d8-494e-bfab-13c54ba1d97c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:51.022171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:51.023072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:51.024489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 220/443] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:51.025136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8d92b-04d8-494e-bfab-13c54ba1d97c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.037862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b682cc-bb87-4bd5-8ff1-40cd457181cc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.074284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432750ac-ea25-45c2-b993-74f781fb1cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.074893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432750ac-ea25-45c2-b993-74f781fb1cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.075202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432750ac-ea25-45c2-b993-74f781fb1cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.075477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432750ac-ea25-45c2-b993-74f781fb1cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.076139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432750ac-ea25-45c2-b993-74f781fb1cc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:51.076439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432750ac-ea25-45c2-b993-74f781fb1cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.077810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432750ac-ea25-45c2-b993-74f781fb1cc9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.085878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 224/444] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:51.086661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432750ac-ea25-45c2-b993-74f781fb1cc9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.126914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8d92b-04d8-494e-bfab-13c54ba1d97c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.138435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:51.159990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ad03e-3ff3-4615-b3f2-5823870514a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.160608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ad03e-3ff3-4615-b3f2-5823870514a1 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.160886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ad03e-3ff3-4615-b3f2-5823870514a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.161155 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ad03e-3ff3-4615-b3f2-5823870514a1 None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.162262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ad03e-3ff3-4615-b3f2-5823870514a1 None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.163744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ad03e-3ff3-4615-b3f2-5823870514a1 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.167684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 221/445] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:51.167684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ad03e-3ff3-4615-b3f2-5823870514a1 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.186936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432750ac-ea25-45c2-b993-74f781fb1cc9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.198807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:51.220454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6d7cf-d186-4f88-939a-d88191670842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.221056 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6d7cf-d186-4f88-939a-d88191670842 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.221337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6d7cf-d186-4f88-939a-d88191670842 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.221610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6d7cf-d186-4f88-939a-d88191670842 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.222269 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6d7cf-d186-4f88-939a-d88191670842 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:51.222575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6d7cf-d186-4f88-939a-d88191670842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.223963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6d7cf-d186-4f88-939a-d88191670842 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.231576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 225/446] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:51.232345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6d7cf-d186-4f88-939a-d88191670842 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.267239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ad03e-3ff3-4615-b3f2-5823870514a1 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.298531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fb715-9fb8-49bc-8d71-b87ac7fb8fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.299162 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fb715-9fb8-49bc-8d71-b87ac7fb8fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.299440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fb715-9fb8-49bc-8d71-b87ac7fb8fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.299711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fb715-9fb8-49bc-8d71-b87ac7fb8fe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.330422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6d7cf-d186-4f88-939a-d88191670842 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.343648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:51.363559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3f9c8-a5d3-4f7b-a302-48f7bb40d4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.364166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3f9c8-a5d3-4f7b-a302-48f7bb40d4dc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.364446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3f9c8-a5d3-4f7b-a302-48f7bb40d4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.364715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3f9c8-a5d3-4f7b-a302-48f7bb40d4dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.365436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3f9c8-a5d3-4f7b-a302-48f7bb40d4dc None admin] RBAC: Adding query filter params (group.id=01363f1eb6934ebf88c6f0b95ff1bb3c, role.id=7694dcfc7d6547abb6565cb9e726eacd, scope.project.id=d254037584f44b54b09c2b3e76807918) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:51.365586 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fb715-9fb8-49bc-8d71-b87ac7fb8fe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.365743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3f9c8-a5d3-4f7b-a302-48f7bb40d4dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.367056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fb715-9fb8-49bc-8d71-b87ac7fb8fe9 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.367655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3f9c8-a5d3-4f7b-a302-48f7bb40d4dc None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.373770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 226/447] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/role_assignments?group.id=01363f1eb6934ebf88c6f0b95ff1bb3c&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=7694dcfc7d6547abb6565cb9e726eacd => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:51.374466 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3f9c8-a5d3-4f7b-a302-48f7bb40d4dc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.415170 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:51.488440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3f9c8-a5d3-4f7b-a302-48f7bb40d4dc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.495600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 222/448] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:51.501550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fb715-9fb8-49bc-8d71-b87ac7fb8fe9 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.531996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957858ca-99f1-45e6-a1a2-a3ff92f9e46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.532541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957858ca-99f1-45e6-a1a2-a3ff92f9e46b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.532923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957858ca-99f1-45e6-a1a2-a3ff92f9e46b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.533095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957858ca-99f1-45e6-a1a2-a3ff92f9e46b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.584406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:51.607564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957858ca-99f1-45e6-a1a2-a3ff92f9e46b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.609095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957858ca-99f1-45e6-a1a2-a3ff92f9e46b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.619735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fb715-9fb8-49bc-8d71-b87ac7fb8fe9 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.655172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:51.661014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970c7c3-49fd-4a1e-b65b-3c69a29732cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.661643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970c7c3-49fd-4a1e-b65b-3c69a29732cf None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.662235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970c7c3-49fd-4a1e-b65b-3c69a29732cf None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.662235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970c7c3-49fd-4a1e-b65b-3c69a29732cf None admin] PATH_INFO: `/v3/projects/9cee49555c8143ed868a4be75b65ed07` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.663365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970c7c3-49fd-4a1e-b65b-3c69a29732cf None admin] RBAC: Authorizing `identity:get_project(project_id=9cee49555c8143ed868a4be75b65ed07)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.666070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970c7c3-49fd-4a1e-b65b-3c69a29732cf None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.667165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 223/449] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/projects/9cee49555c8143ed868a4be75b65ed07 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:51.672605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970c7c3-49fd-4a1e-b65b-3c69a29732cf None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.730441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 227/450] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:50:51.736133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957858ca-99f1-45e6-a1a2-a3ff92f9e46b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.780627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970c7c3-49fd-4a1e-b65b-3c69a29732cf None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.812283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:51.814409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed728ee-17c0-4e45-aea6-3d3280dfd71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.815065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed728ee-17c0-4e45-aea6-3d3280dfd71d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.815339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed728ee-17c0-4e45-aea6-3d3280dfd71d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.815605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed728ee-17c0-4e45-aea6-3d3280dfd71d None admin] PATH_INFO: `/v3/groups/590a9b9a5cea4037b31e3cd2e37d7f1c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.816714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed728ee-17c0-4e45-aea6-3d3280dfd71d None admin] RBAC: Authorizing `identity:get_group(group_id=590a9b9a5cea4037b31e3cd2e37d7f1c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.818182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed728ee-17c0-4e45-aea6-3d3280dfd71d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.819787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 224/451] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/groups/590a9b9a5cea4037b31e3cd2e37d7f1c => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:51.825644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed728ee-17c0-4e45-aea6-3d3280dfd71d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.834712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957858ca-99f1-45e6-a1a2-a3ff92f9e46b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.869964 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448907f2-f666-41e8-93ae-ed8488e7eaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.870593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448907f2-f666-41e8-93ae-ed8488e7eaff None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.870876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448907f2-f666-41e8-93ae-ed8488e7eaff None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.871183 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448907f2-f666-41e8-93ae-ed8488e7eaff None admin] PATH_INFO: `/v3/projects/d254037584f44b54b09c2b3e76807918` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.872296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448907f2-f666-41e8-93ae-ed8488e7eaff None admin] RBAC: Authorizing `identity:get_project(project_id=d254037584f44b54b09c2b3e76807918)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.873714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448907f2-f666-41e8-93ae-ed8488e7eaff None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.875360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 228/452] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/projects/d254037584f44b54b09c2b3e76807918 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:51.880883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448907f2-f666-41e8-93ae-ed8488e7eaff None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:51.890481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:51.923964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed728ee-17c0-4e45-aea6-3d3280dfd71d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:51.956239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164f151-3e4e-4988-bba5-986dd05d2ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:51.956829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164f151-3e4e-4988-bba5-986dd05d2ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:51.957104 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164f151-3e4e-4988-bba5-986dd05d2ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:51.957380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164f151-3e4e-4988-bba5-986dd05d2ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:51.958100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164f151-3e4e-4988-bba5-986dd05d2ef0 None admin] RBAC: Adding query filter params (group.id=590a9b9a5cea4037b31e3cd2e37d7f1c, role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=9cee49555c8143ed868a4be75b65ed07) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:51.958406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164f151-3e4e-4988-bba5-986dd05d2ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:51.959879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164f151-3e4e-4988-bba5-986dd05d2ef0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:51.966119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 225/453] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/role_assignments?group.id=590a9b9a5cea4037b31e3cd2e37d7f1c&scope.project.id=9cee49555c8143ed868a4be75b65ed07&role.id=8417343e9ef64991bcf51bb9692da102 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:51.979118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448907f2-f666-41e8-93ae-ed8488e7eaff None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:52.016241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08053501-55c5-4ec2-9ed3-ba4594b54585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:52.016893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08053501-55c5-4ec2-9ed3-ba4594b54585 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:52.017176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08053501-55c5-4ec2-9ed3-ba4594b54585 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:52.017449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08053501-55c5-4ec2-9ed3-ba4594b54585 None admin] PATH_INFO: `/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:52.018651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08053501-55c5-4ec2-9ed3-ba4594b54585 None admin] RBAC: Authorizing `identity:get_group(group_id=01363f1eb6934ebf88c6f0b95ff1bb3c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:52.023137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08053501-55c5-4ec2-9ed3-ba4594b54585 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:52.023137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 229/454] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 10 09:50:51 2022] GET /identity/v3/groups/01363f1eb6934ebf88c6f0b95ff1bb3c => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:52.028567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164f151-3e4e-4988-bba5-986dd05d2ef0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:52.135783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164f151-3e4e-4988-bba5-986dd05d2ef0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:52.164164 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:52.169580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86114f44-0fe1-4b85-8d17-38bdece4dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:52.170179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86114f44-0fe1-4b85-8d17-38bdece4dd82 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:52.170455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86114f44-0fe1-4b85-8d17-38bdece4dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:52.170725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86114f44-0fe1-4b85-8d17-38bdece4dd82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:52.171485 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86114f44-0fe1-4b85-8d17-38bdece4dd82 None admin] RBAC: Adding query filter params (group.id=01363f1eb6934ebf88c6f0b95ff1bb3c, role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=d254037584f44b54b09c2b3e76807918) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:52.171795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86114f44-0fe1-4b85-8d17-38bdece4dd82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:52.173226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86114f44-0fe1-4b85-8d17-38bdece4dd82 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:52.178855 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 226/455] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 10 09:50:52 2022] GET /identity/v3/role_assignments?group.id=01363f1eb6934ebf88c6f0b95ff1bb3c&scope.project.id=d254037584f44b54b09c2b3e76807918&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:54.845669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c7781-c641-465f-920f-5e13ae0c4411 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:54.846215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c7781-c641-465f-920f-5e13ae0c4411 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:54.846215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c7781-c641-465f-920f-5e13ae0c4411 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:54.847856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 230/456] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:54.853214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42910d-5d74-4b52-a35a-ea61284e5e52 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:54.853616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42910d-5d74-4b52-a35a-ea61284e5e52 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:54.853785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42910d-5d74-4b52-a35a-ea61284e5e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:54.880225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ff42910d-5d74-4b52-a35a-ea61284e5e52 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:54.881722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ff42910d-5d74-4b52-a35a-ea61284e5e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:54.930887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17359d9-e3a5-43f4-b235-5250546578cd None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:54.931524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17359d9-e3a5-43f4-b235-5250546578cd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:54.931994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17359d9-e3a5-43f4-b235-5250546578cd None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:54.934890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 231/457] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:54.939186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25765d-6169-49b7-b599-995a0a8fa9ad None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:54.939695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25765d-6169-49b7-b599-995a0a8fa9ad None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:54.940195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25765d-6169-49b7-b599-995a0a8fa9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:54.960305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ba25765d-6169-49b7-b599-995a0a8fa9ad None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:54.962339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ba25765d-6169-49b7-b599-995a0a8fa9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:54.979265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 227/458] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:54.988071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:55.020962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42910d-5d74-4b52-a35a-ea61284e5e52 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:55.039991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 232/459] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:55.061546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25765d-6169-49b7-b599-995a0a8fa9ad None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:55.176667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42910d-5d74-4b52-a35a-ea61284e5e52 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:55.183404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25765d-6169-49b7-b599-995a0a8fa9ad None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:55.206029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:55.217158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230742c-9e26-4b07-a5cf-f5fb8b9e1c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:55.218033 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230742c-9e26-4b07-a5cf-f5fb8b9e1c25 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.218482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230742c-9e26-4b07-a5cf-f5fb8b9e1c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.218900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230742c-9e26-4b07-a5cf-f5fb8b9e1c25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.225959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3485a4-387f-4590-9e9d-b3787eaffe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:55.226809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3485a4-387f-4590-9e9d-b3787eaffe8c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.227239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3485a4-387f-4590-9e9d-b3787eaffe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.228280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3485a4-387f-4590-9e9d-b3787eaffe8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.232383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230742c-9e26-4b07-a5cf-f5fb8b9e1c25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:55.234052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230742c-9e26-4b07-a5cf-f5fb8b9e1c25 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:55.240601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3485a4-387f-4590-9e9d-b3787eaffe8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:55.242727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3485a4-387f-4590-9e9d-b3787eaffe8c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:55.246378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-6230742c-9e26-4b07-a5cf-f5fb8b9e1c25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:50:55.252867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:55.253786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:50:55.254877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:50:55.254877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:55.254877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:55.257148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 228/460] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:55.259223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac2de7-ec3c-40cb-ae72-70d130f2c26c None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.259682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac2de7-ec3c-40cb-ae72-70d130f2c26c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.260113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac2de7-ec3c-40cb-ae72-70d130f2c26c None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.261578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 229/461] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:55.263313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac2de7-ec3c-40cb-ae72-70d130f2c26c None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-2d3485a4-387f-4590-9e9d-b3787eaffe8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:50:55.265828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:55.266709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:50:55.267626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:50:55.267626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:55.267626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:55.271367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 233/462] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:55.274321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179ec1a-6c15-4281-9b0f-6f052ebb46ac None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.274604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179ec1a-6c15-4281-9b0f-6f052ebb46ac None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.274876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179ec1a-6c15-4281-9b0f-6f052ebb46ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.298410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2179ec1a-6c15-4281-9b0f-6f052ebb46ac None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:55.300257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2179ec1a-6c15-4281-9b0f-6f052ebb46ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:55.381904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 234/463] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:55.383654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179ec1a-6c15-4281-9b0f-6f052ebb46ac None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:55.420983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac2de7-ec3c-40cb-ae72-70d130f2c26c None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:55.460124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043ae0d-818d-4ea6-95b7-d652c97ac31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:55.461642 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043ae0d-818d-4ea6-95b7-d652c97ac31b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.462110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043ae0d-818d-4ea6-95b7-d652c97ac31b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.462560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043ae0d-818d-4ea6-95b7-d652c97ac31b None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.463359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043ae0d-818d-4ea6-95b7-d652c97ac31b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:55.463810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043ae0d-818d-4ea6-95b7-d652c97ac31b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:55.465361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043ae0d-818d-4ea6-95b7-d652c97ac31b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:55.482720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 230/464] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:55.482720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043ae0d-818d-4ea6-95b7-d652c97ac31b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:55.510810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179ec1a-6c15-4281-9b0f-6f052ebb46ac None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:55.524874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:55.537751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:55.547867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50e663-b3c6-4370-9980-0ce4ca537635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:55.548406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50e663-b3c6-4370-9980-0ce4ca537635 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.548769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50e663-b3c6-4370-9980-0ce4ca537635 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.548962 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50e663-b3c6-4370-9980-0ce4ca537635 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.549596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50e663-b3c6-4370-9980-0ce4ca537635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:55.549899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50e663-b3c6-4370-9980-0ce4ca537635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:55.551299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50e663-b3c6-4370-9980-0ce4ca537635 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:55.563011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 235/465] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:55.564193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883ca2d-b611-4b41-961c-7b52c0b0b9d2 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.564480 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883ca2d-b611-4b41-961c-7b52c0b0b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.564769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883ca2d-b611-4b41-961c-7b52c0b0b9d2 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.566083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 236/466] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:55.566626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883ca2d-b611-4b41-961c-7b52c0b0b9d2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:55.596086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043ae0d-818d-4ea6-95b7-d652c97ac31b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:55.632068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375729d8-2c45-4d76-b663-d6a243fa0120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:55.633492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375729d8-2c45-4d76-b663-d6a243fa0120 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.633761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375729d8-2c45-4d76-b663-d6a243fa0120 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.634020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375729d8-2c45-4d76-b663-d6a243fa0120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.644802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375729d8-2c45-4d76-b663-d6a243fa0120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:55.646265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375729d8-2c45-4d76-b663-d6a243fa0120 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-375729d8-2c45-4d76-b663-d6a243fa0120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:50:55.660486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:55.661364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:50:55.662206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:50:55.662206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:55.662206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:55.662206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 231/467] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:55.662418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375729d8-2c45-4d76-b663-d6a243fa0120 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:55.675909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883ca2d-b611-4b41-961c-7b52c0b0b9d2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:55.712437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548d552-02b5-4577-94d2-d89d6ead2237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:55.713435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548d552-02b5-4577-94d2-d89d6ead2237 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.713929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548d552-02b5-4577-94d2-d89d6ead2237 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.714387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548d552-02b5-4577-94d2-d89d6ead2237 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.715207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548d552-02b5-4577-94d2-d89d6ead2237 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:55.717071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548d552-02b5-4577-94d2-d89d6ead2237 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:55.740390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 237/468] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:55 2022] POST /identity/v3/users => generated 248 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:55.741956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd18ef-5116-49c6-a297-7f3814a49573 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.742989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd18ef-5116-49c6-a297-7f3814a49573 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.743861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd18ef-5116-49c6-a297-7f3814a49573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.763895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7cdd18ef-5116-49c6-a297-7f3814a49573 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:55.765107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7cdd18ef-5116-49c6-a297-7f3814a49573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:55.766533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:55.782300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375729d8-2c45-4d76-b663-d6a243fa0120 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:55.809253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:55.837329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe423192-1a3d-4db6-9a53-4f022d30c061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:55.838937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe423192-1a3d-4db6-9a53-4f022d30c061 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:55.839527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe423192-1a3d-4db6-9a53-4f022d30c061 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:55.840053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe423192-1a3d-4db6-9a53-4f022d30c061 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:55.841178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe423192-1a3d-4db6-9a53-4f022d30c061 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:55.843631 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe423192-1a3d-4db6-9a53-4f022d30c061 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:55.849955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 238/469] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:55.855601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd18ef-5116-49c6-a297-7f3814a49573 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:55.872712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 232/470] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:55 2022] POST /identity/v3/users => generated 246 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:55.890246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe423192-1a3d-4db6-9a53-4f022d30c061 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:56.032505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe423192-1a3d-4db6-9a53-4f022d30c061 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:56.036718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd18ef-5116-49c6-a297-7f3814a49573 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:56.092035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c8d8b-441e-412f-8e83-0a3a1609acb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:56.092993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5d92-a056-46a1-8e85-c3bf075b8db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:56.093856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c8d8b-441e-412f-8e83-0a3a1609acb6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:56.094684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5d92-a056-46a1-8e85-c3bf075b8db0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:56.095206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c8d8b-441e-412f-8e83-0a3a1609acb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:56.095381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5d92-a056-46a1-8e85-c3bf075b8db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:56.095642 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5d92-a056-46a1-8e85-c3bf075b8db0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:56.096050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c8d8b-441e-412f-8e83-0a3a1609acb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:56.096890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c8d8b-441e-412f-8e83-0a3a1609acb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:56.097386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c8d8b-441e-412f-8e83-0a3a1609acb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:56.099061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c8d8b-441e-412f-8e83-0a3a1609acb6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:56.109675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b5d92-a056-46a1-8e85-c3bf075b8db0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:56.111420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b5d92-a056-46a1-8e85-c3bf075b8db0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:56.112934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 239/471] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:56.118835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c8d8b-441e-412f-8e83-0a3a1609acb6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:56.119419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-025b5d92-a056-46a1-8e85-c3bf075b8db0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:50:56.129275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:56.130234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:50:56.131115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:50:56.131115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:56.131115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:56.134794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 233/472] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:56.141996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5d92-a056-46a1-8e85-c3bf075b8db0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:56.159461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:56.250046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c8d8b-441e-412f-8e83-0a3a1609acb6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:56.277615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5d92-a056-46a1-8e85-c3bf075b8db0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:56.283327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765a4d0-4e68-4ac3-b023-49cb32580b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:56.283952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765a4d0-4e68-4ac3-b023-49cb32580b6d None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:56.284302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765a4d0-4e68-4ac3-b023-49cb32580b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:56.284643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765a4d0-4e68-4ac3-b023-49cb32580b6d None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:56.285355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765a4d0-4e68-4ac3-b023-49cb32580b6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:56.286868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765a4d0-4e68-4ac3-b023-49cb32580b6d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:56.313205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 240/473] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:56 2022] POST /identity/v3/users => generated 248 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:56.314651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701ec63-214b-4b60-acca-f03baf67d05a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:56.314878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701ec63-214b-4b60-acca-f03baf67d05a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:56.315173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701ec63-214b-4b60-acca-f03baf67d05a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:56.315868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8723b1-ebfe-4237-8198-646032774c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:56.316480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8723b1-ebfe-4237-8198-646032774c5a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:56.316756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8723b1-ebfe-4237-8198-646032774c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:56.317025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8723b1-ebfe-4237-8198-646032774c5a None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:56.317735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8723b1-ebfe-4237-8198-646032774c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:56.318038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8723b1-ebfe-4237-8198-646032774c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:56.319527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8723b1-ebfe-4237-8198-646032774c5a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:56.320971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 241/474] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:56.323163 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621d674-b0e8-4bfd-8163-9c8dedb870ff None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:56.323456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621d674-b0e8-4bfd-8163-9c8dedb870ff None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:56.323712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621d674-b0e8-4bfd-8163-9c8dedb870ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:56.333560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 234/475] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:56.344123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8723b1-ebfe-4237-8198-646032774c5a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:56.345967 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e621d674-b0e8-4bfd-8163-9c8dedb870ff None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:56.347234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e621d674-b0e8-4bfd-8163-9c8dedb870ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:56.464798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:56.475061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 242/476] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:56.494624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621d674-b0e8-4bfd-8163-9c8dedb870ff None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:56.514476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:56.541496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8723b1-ebfe-4237-8198-646032774c5a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:56.585588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a331219-2b6e-45ef-94bd-d64ce689b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:56.587429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331219-2b6e-45ef-94bd-d64ce689b3b3 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:56.587850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331219-2b6e-45ef-94bd-d64ce689b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:56.588259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331219-2b6e-45ef-94bd-d64ce689b3b3 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:56.589062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a331219-2b6e-45ef-94bd-d64ce689b3b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:56.591033 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a331219-2b6e-45ef-94bd-d64ce689b3b3 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:56.617662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621d674-b0e8-4bfd-8163-9c8dedb870ff None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:56.619918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 235/477] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:50:56 2022] POST /identity/v3/users => generated 249 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:50:56.660079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:56.660711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:56.660997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:56.661272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:56.661956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:56.663420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:56.681251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.sql.core [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 09:50:56.681251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 09:50:56.681251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [parameters: {'id': '6d7e44ce9ab44b5dba8ee115c3e960f2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 09:50:56.681251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80247) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application conn.query(q) Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result.read() Aug 10 09:50:56.698819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 09:50:56.699776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application next(self.gen) Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application yield resource Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application next(self.gen) Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise self.value Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application session.commit() Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 09:50:56.700935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.session.flush() Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application compat.raise_( Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 09:50:56.701791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 09:50:56.702662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application conn.query(q) Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result.read() Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 09:50:56.703511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [parameters: {'id': '6d7e44ce9ab44b5dba8ee115c3e960f2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:56.704463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 09:50:56.705233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:56.706048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 243/478] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 10 09:50:56 2022] POST /identity/v3/roles => generated 144 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:56.709766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a331219-2b6e-45ef-94bd-d64ce689b3b3 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:56.866629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a331219-2b6e-45ef-94bd-d64ce689b3b3 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:56.881458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:56.908064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b2032-885c-4121-a461-04b7a45ed0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:56.908706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b2032-885c-4121-a461-04b7a45ed0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:56.908985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b2032-885c-4121-a461-04b7a45ed0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:56.909259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b2032-885c-4121-a461-04b7a45ed0c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:56.925919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b2032-885c-4121-a461-04b7a45ed0c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-c48b2032-885c-4121-a461-04b7a45ed0c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:56.931278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:56.932158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:56.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 236/479] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:50:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:56.940477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:57.061584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:57.090793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be571f-57be-4f77-8ef4-187c939e621f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:57.140828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884bfc2-add4-4d86-b1a6-66676e6b147d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:57.141810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884bfc2-add4-4d86-b1a6-66676e6b147d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:57.142299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884bfc2-add4-4d86-b1a6-66676e6b147d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:57.142768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884bfc2-add4-4d86-b1a6-66676e6b147d None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:57.143721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884bfc2-add4-4d86-b1a6-66676e6b147d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:57.144321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884bfc2-add4-4d86-b1a6-66676e6b147d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:57.145909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884bfc2-add4-4d86-b1a6-66676e6b147d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:57.159592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 244/480] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:50:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:58.202397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eba457-5fe1-4f4f-828d-bc5e04608843 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:58.202914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eba457-5fe1-4f4f-828d-bc5e04608843 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:58.202914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eba457-5fe1-4f4f-828d-bc5e04608843 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:58.204553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 237/481] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:58.209615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f503916-69ad-42c2-b7bc-c8c3ca5f5c95 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:58.210022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f503916-69ad-42c2-b7bc-c8c3ca5f5c95 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:58.210189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f503916-69ad-42c2-b7bc-c8c3ca5f5c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:58.235928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1f503916-69ad-42c2-b7bc-c8c3ca5f5c95 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:58.237116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1f503916-69ad-42c2-b7bc-c8c3ca5f5c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:58.333843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 245/482] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:58.368877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eba457-5fe1-4f4f-828d-bc5e04608843 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:58.511707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eba457-5fe1-4f4f-828d-bc5e04608843 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:58.534216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fd0ab-24da-41d2-851b-d2b177795ff6 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:58.534495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fd0ab-24da-41d2-851b-d2b177795ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:58.534766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fd0ab-24da-41d2-851b-d2b177795ff6 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:58.536365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 246/483] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:58.541817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdd789-5d52-4e91-9f00-5abbf6e15ef2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:58.542092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdd789-5d52-4e91-9f00-5abbf6e15ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:58.542362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdd789-5d52-4e91-9f00-5abbf6e15ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:58.559634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484cc66-d22c-451c-8dbf-14b2df11d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:58.560269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484cc66-d22c-451c-8dbf-14b2df11d4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:58.560550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484cc66-d22c-451c-8dbf-14b2df11d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:58.560654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-31fdd789-5d52-4e91-9f00-5abbf6e15ef2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:58.560827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484cc66-d22c-451c-8dbf-14b2df11d4c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:58.561920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-31fdd789-5d52-4e91-9f00-5abbf6e15ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:58.579775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484cc66-d22c-451c-8dbf-14b2df11d4c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-f484cc66-d22c-451c-8dbf-14b2df11d4c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:58.584841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:58.585707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:58.587065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 238/484] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:58.592925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484cc66-d22c-451c-8dbf-14b2df11d4c6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:58.650390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 247/485] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:50:58.663452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:58.752407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdd789-5d52-4e91-9f00-5abbf6e15ef2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:58.752746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484cc66-d22c-451c-8dbf-14b2df11d4c6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:58.760095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:58.788759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb31fe-d431-45a5-abc1-fa707852c2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:58.789857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb31fe-d431-45a5-abc1-fa707852c2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:58.790130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb31fe-d431-45a5-abc1-fa707852c2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:58.790398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb31fe-d431-45a5-abc1-fa707852c2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:58.791096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb31fe-d431-45a5-abc1-fa707852c2a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:58.791392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb31fe-d431-45a5-abc1-fa707852c2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:58.792771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb31fe-d431-45a5-abc1-fa707852c2a0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:58.802372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 239/486] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:50:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:50:58.807856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb31fe-d431-45a5-abc1-fa707852c2a0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:58.909286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdd789-5d52-4e91-9f00-5abbf6e15ef2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:58.913230 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb31fe-d431-45a5-abc1-fa707852c2a0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:58.916861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:58.944350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765ffcb-6a58-4434-afc6-5d7dcc196cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:58.945631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765ffcb-6a58-4434-afc6-5d7dcc196cdd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:58.946050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765ffcb-6a58-4434-afc6-5d7dcc196cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:58.946440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765ffcb-6a58-4434-afc6-5d7dcc196cdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:58.955609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d39517-3585-4720-b849-b5914a031a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:58.956499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d39517-3585-4720-b849-b5914a031a08 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:58.956776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d39517-3585-4720-b849-b5914a031a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:58.956956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765ffcb-6a58-4434-afc6-5d7dcc196cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:58.957077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d39517-3585-4720-b849-b5914a031a08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-2765ffcb-6a58-4434-afc6-5d7dcc196cdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:58.962492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:58.963457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:50:58.964245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:58.964853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 248/487] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:58.968864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd57c1-3cc6-4848-9a90-2a6facce59c9 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:58.968864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd57c1-3cc6-4848-9a90-2a6facce59c9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:58.968864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd57c1-3cc6-4848-9a90-2a6facce59c9 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:58.968864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 249/488] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:58.970079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd57c1-3cc6-4848-9a90-2a6facce59c9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:58.972654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d39517-3585-4720-b849-b5914a031a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:58.974301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d39517-3585-4720-b849-b5914a031a08 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-66d39517-3585-4720-b849-b5914a031a08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:50:59.003791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:59.005002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:50:59.006056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:50:59.006056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:59.006056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:59.011010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 240/489] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:50:59.012433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195966b-05d9-4837-9e50-9fa970098595 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.012708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195966b-05d9-4837-9e50-9fa970098595 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.012977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195966b-05d9-4837-9e50-9fa970098595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.032175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6195966b-05d9-4837-9e50-9fa970098595 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:59.033399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6195966b-05d9-4837-9e50-9fa970098595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:59.050318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:59.102574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 241/490] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:59.103535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195966b-05d9-4837-9e50-9fa970098595 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.106278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd57c1-3cc6-4848-9a90-2a6facce59c9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:59.151259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcc1b6-e475-42f8-a46b-a3e175a60898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:59.151879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcc1b6-e475-42f8-a46b-a3e175a60898 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.152158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcc1b6-e475-42f8-a46b-a3e175a60898 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.152427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcc1b6-e475-42f8-a46b-a3e175a60898 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.153107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcc1b6-e475-42f8-a46b-a3e175a60898 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:59.153437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcc1b6-e475-42f8-a46b-a3e175a60898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:59.154820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcc1b6-e475-42f8-a46b-a3e175a60898 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:59.163995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 250/491] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:50:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:59.168578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcc1b6-e475-42f8-a46b-a3e175a60898 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.200938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:59.229553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195966b-05d9-4837-9e50-9fa970098595 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:59.268828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bc71c-cac7-4015-b9ea-e8d3351f3d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:59.269638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bc71c-cac7-4015-b9ea-e8d3351f3d8c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.270011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bc71c-cac7-4015-b9ea-e8d3351f3d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.270421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bc71c-cac7-4015-b9ea-e8d3351f3d8c None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.271200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bc71c-cac7-4015-b9ea-e8d3351f3d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:59.271615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bc71c-cac7-4015-b9ea-e8d3351f3d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:59.273854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bc71c-cac7-4015-b9ea-e8d3351f3d8c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:59.285951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 242/492] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:59.286875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcc1b6-e475-42f8-a46b-a3e175a60898 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:59.287359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bc71c-cac7-4015-b9ea-e8d3351f3d8c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.321330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:59.331038 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10cdb1-da2a-4217-be02-82a02442df44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:59.331668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10cdb1-da2a-4217-be02-82a02442df44 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.332236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10cdb1-da2a-4217-be02-82a02442df44 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.332236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10cdb1-da2a-4217-be02-82a02442df44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.343373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10cdb1-da2a-4217-be02-82a02442df44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-4f10cdb1-da2a-4217-be02-82a02442df44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:50:59.348245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:59.349122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:50:59.349898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:59.354276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 251/493] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:59.355139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10cdb1-da2a-4217-be02-82a02442df44 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.395329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bc71c-cac7-4015-b9ea-e8d3351f3d8c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:59.436124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a682f8-7843-4638-8525-584dbf41f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:59.436752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a682f8-7843-4638-8525-584dbf41f656 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.437032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a682f8-7843-4638-8525-584dbf41f656 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.437300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a682f8-7843-4638-8525-584dbf41f656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.444551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:59.461401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a682f8-7843-4638-8525-584dbf41f656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:59.462849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a682f8-7843-4638-8525-584dbf41f656 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:59.466882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10cdb1-da2a-4217-be02-82a02442df44 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-21a682f8-7843-4638-8525-584dbf41f656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:50:59.482201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:59.483177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:50:59.484019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:50:59.484019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:59.484019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:59.484019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 243/494] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:59.484879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaab083-56d6-448e-b159-1c9ada516ede None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.484879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaab083-56d6-448e-b159-1c9ada516ede None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.485048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaab083-56d6-448e-b159-1c9ada516ede None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.486357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 244/495] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:59.486906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaab083-56d6-448e-b159-1c9ada516ede None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.515351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12cec7-2c70-4152-910d-9d6878fcfe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:59.515991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12cec7-2c70-4152-910d-9d6878fcfe36 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.516269 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12cec7-2c70-4152-910d-9d6878fcfe36 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.516541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12cec7-2c70-4152-910d-9d6878fcfe36 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.524118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12cec7-2c70-4152-910d-9d6878fcfe36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:59.525580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12cec7-2c70-4152-910d-9d6878fcfe36 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-7b12cec7-2c70-4152-910d-9d6878fcfe36 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:50:59.538384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:59.539300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 09:50:59.540192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:50:59.544206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 252/496] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:59.545060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12cec7-2c70-4152-910d-9d6878fcfe36 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.591541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:59.593841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaab083-56d6-448e-b159-1c9ada516ede None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:59.627975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c33ff8-ccc6-4b22-96c1-347c92525b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:59.628572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33ff8-ccc6-4b22-96c1-347c92525b25 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.628844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33ff8-ccc6-4b22-96c1-347c92525b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.629110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33ff8-ccc6-4b22-96c1-347c92525b25 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.629753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c33ff8-ccc6-4b22-96c1-347c92525b25 None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:59.630049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c33ff8-ccc6-4b22-96c1-347c92525b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:59.631446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c33ff8-ccc6-4b22-96c1-347c92525b25 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:59.640450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 245/497] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:59.641459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3697ae1-86ed-4f3c-b034-67d95a4eb5d2 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.641744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3697ae1-86ed-4f3c-b034-67d95a4eb5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.642008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3697ae1-86ed-4f3c-b034-67d95a4eb5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.652218 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12cec7-2c70-4152-910d-9d6878fcfe36 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:59.659047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a3697ae1-86ed-4f3c-b034-67d95a4eb5d2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:50:59.660194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a3697ae1-86ed-4f3c-b034-67d95a4eb5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:50:59.687412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59e957-8077-47b9-be74-7205d4b46271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:59.688015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59e957-8077-47b9-be74-7205d4b46271 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.688296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59e957-8077-47b9-be74-7205d4b46271 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.688809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59e957-8077-47b9-be74-7205d4b46271 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.689204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59e957-8077-47b9-be74-7205d4b46271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:59.689508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59e957-8077-47b9-be74-7205d4b46271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:59.690880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59e957-8077-47b9-be74-7205d4b46271 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:59.703837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 253/498] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:59.704568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59e957-8077-47b9-be74-7205d4b46271 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.720032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:59.741853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 246/499] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:50:59.743534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3697ae1-86ed-4f3c-b034-67d95a4eb5d2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.807885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59e957-8077-47b9-be74-7205d4b46271 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:59.817213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:59.843256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7388900-2d8d-4744-8311-c527ddc04729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:59.843784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7388900-2d8d-4744-8311-c527ddc04729 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.844058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7388900-2d8d-4744-8311-c527ddc04729 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.844162 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3697ae1-86ed-4f3c-b034-67d95a4eb5d2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:50:59.844325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7388900-2d8d-4744-8311-c527ddc04729 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.845033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7388900-2d8d-4744-8311-c527ddc04729 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:50:59.845333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7388900-2d8d-4744-8311-c527ddc04729 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:59.846760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7388900-2d8d-4744-8311-c527ddc04729 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:59.877753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf7ca2-e7ef-427d-9446-7a748dfc0882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:50:59.878376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf7ca2-e7ef-427d-9446-7a748dfc0882 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.878653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf7ca2-e7ef-427d-9446-7a748dfc0882 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.878923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf7ca2-e7ef-427d-9446-7a748dfc0882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.883822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 254/500] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:50:59.884651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7388900-2d8d-4744-8311-c527ddc04729 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.891877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf7ca2-e7ef-427d-9446-7a748dfc0882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:50:59.893334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf7ca2-e7ef-427d-9446-7a748dfc0882 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-a0bf7ca2-e7ef-427d-9446-7a748dfc0882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:50:59.907418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:50:59.908344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:50:59.909402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:50:59.909402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:50:59.909402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:50:59.909402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 247/501] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:50:59.910039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25c77-5c11-4fbc-8fcf-821418db49a0 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:50:59.910039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25c77-5c11-4fbc-8fcf-821418db49a0 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:50:59.910189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25c77-5c11-4fbc-8fcf-821418db49a0 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:50:59.911388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 248/502] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:50:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:50:59.912080 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25c77-5c11-4fbc-8fcf-821418db49a0 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:50:59.946848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:50:59.986659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7388900-2d8d-4744-8311-c527ddc04729 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.012658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25c77-5c11-4fbc-8fcf-821418db49a0 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.019785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a08c0c-52ae-43c9-b4e1-2d1c1310e89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.020432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a08c0c-52ae-43c9-b4e1-2d1c1310e89d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.020710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a08c0c-52ae-43c9-b4e1-2d1c1310e89d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.020982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a08c0c-52ae-43c9-b4e1-2d1c1310e89d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.027752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a08c0c-52ae-43c9-b4e1-2d1c1310e89d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.029237 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a08c0c-52ae-43c9-b4e1-2d1c1310e89d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-a3a08c0c-52ae-43c9-b4e1-2d1c1310e89d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:00.040968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:00.041850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:00.042720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 255/503] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:00.043316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a08c0c-52ae-43c9-b4e1-2d1c1310e89d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.048276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8ec11-6359-4f0c-9e77-d0ee8f6ce4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.048902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8ec11-6359-4f0c-9e77-d0ee8f6ce4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.049181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8ec11-6359-4f0c-9e77-d0ee8f6ce4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.049451 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8ec11-6359-4f0c-9e77-d0ee8f6ce4a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.060091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8ec11-6359-4f0c-9e77-d0ee8f6ce4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-6da8ec11-6359-4f0c-9e77-d0ee8f6ce4a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:00.065107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:00.066106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 249/504] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:50:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:00.067060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8ec11-6359-4f0c-9e77-d0ee8f6ce4a2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.070206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:00.143424 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a08c0c-52ae-43c9-b4e1-2d1c1310e89d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.159311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:00.167191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8ec11-6359-4f0c-9e77-d0ee8f6ce4a2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.176884 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ec76d-c0c5-48f4-a4c3-49b75b3fbb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.177484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ec76d-c0c5-48f4-a4c3-49b75b3fbb95 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.177761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ec76d-c0c5-48f4-a4c3-49b75b3fbb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.178029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ec76d-c0c5-48f4-a4c3-49b75b3fbb95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.200201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6dd08-1895-40f1-bff4-ea76999ee3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.200788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6dd08-1895-40f1-bff4-ea76999ee3dc None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.201058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6dd08-1895-40f1-bff4-ea76999ee3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.201329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6dd08-1895-40f1-bff4-ea76999ee3dc None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.201951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6dd08-1895-40f1-bff4-ea76999ee3dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:00.202251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6dd08-1895-40f1-bff4-ea76999ee3dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.203651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6dd08-1895-40f1-bff4-ea76999ee3dc None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.219095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 250/505] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:00.220497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fb60a-ef58-4a9b-9b73-074d0541c702 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.220785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fb60a-ef58-4a9b-9b73-074d0541c702 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.221066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fb60a-ef58-4a9b-9b73-074d0541c702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.239211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-4d1fb60a-ef58-4a9b-9b73-074d0541c702 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:00.240388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4d1fb60a-ef58-4a9b-9b73-074d0541c702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:00.244697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ec76d-c0c5-48f4-a4c3-49b75b3fbb95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.246156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ec76d-c0c5-48f4-a4c3-49b75b3fbb95 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.269427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:00.305444 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 251/506] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:00.306267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fb60a-ef58-4a9b-9b73-074d0541c702 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.347415 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 256/507] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:00.348252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ec76d-c0c5-48f4-a4c3-49b75b3fbb95 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.392455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:00.411502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fb60a-ef58-4a9b-9b73-074d0541c702 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.445707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c74b4-0319-4f09-976e-ea83d911e42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.445905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ec76d-c0c5-48f4-a4c3-49b75b3fbb95 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.446321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c74b4-0319-4f09-976e-ea83d911e42b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.446601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c74b4-0319-4f09-976e-ea83d911e42b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.446873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c74b4-0319-4f09-976e-ea83d911e42b None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.447624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c74b4-0319-4f09-976e-ea83d911e42b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:00.447932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c74b4-0319-4f09-976e-ea83d911e42b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.449366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c74b4-0319-4f09-976e-ea83d911e42b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.468911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:00.479901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff3bd5-d5d4-4022-97da-3ba5fc762470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.480503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff3bd5-d5d4-4022-97da-3ba5fc762470 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.480781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff3bd5-d5d4-4022-97da-3ba5fc762470 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.481051 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff3bd5-d5d4-4022-97da-3ba5fc762470 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.481716 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff3bd5-d5d4-4022-97da-3ba5fc762470 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:00.482014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff3bd5-d5d4-4022-97da-3ba5fc762470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.483416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff3bd5-d5d4-4022-97da-3ba5fc762470 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.485273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 252/508] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:00.485892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c74b4-0319-4f09-976e-ea83d911e42b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.496052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 257/509] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:00.496857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff3bd5-d5d4-4022-97da-3ba5fc762470 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.585044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c74b4-0319-4f09-976e-ea83d911e42b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.594046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff3bd5-d5d4-4022-97da-3ba5fc762470 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.619383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ede3f3-76c6-4966-a254-264a9ced46a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.620002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ede3f3-76c6-4966-a254-264a9ced46a9 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.620277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ede3f3-76c6-4966-a254-264a9ced46a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.620553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ede3f3-76c6-4966-a254-264a9ced46a9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.627264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ede3f3-76c6-4966-a254-264a9ced46a9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.627746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16274f03-d690-4f98-aea1-f14e3abbf70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.628940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ede3f3-76c6-4966-a254-264a9ced46a9 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.629264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16274f03-d690-4f98-aea1-f14e3abbf70d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.629502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16274f03-d690-4f98-aea1-f14e3abbf70d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.629772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16274f03-d690-4f98-aea1-f14e3abbf70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.639723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16274f03-d690-4f98-aea1-f14e3abbf70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-c1ede3f3-76c6-4966-a254-264a9ced46a9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:00.640955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:00.641976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 09:51:00.642863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:00.643388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16274f03-d690-4f98-aea1-f14e3abbf70d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.643512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 253/510] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:00.643512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ede3f3-76c6-4966-a254-264a9ced46a9 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.645853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-16274f03-d690-4f98-aea1-f14e3abbf70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:00.664541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:00.665883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:00.666727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:00.666727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:00.666727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:00.666727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 258/511] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:00.666942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16274f03-d690-4f98-aea1-f14e3abbf70d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.705687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:00.748214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ede3f3-76c6-4966-a254-264a9ced46a9 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.763375 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16274f03-d690-4f98-aea1-f14e3abbf70d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.779736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40291bdd-ad4e-40e0-a986-b7aa91d62af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.780349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40291bdd-ad4e-40e0-a986-b7aa91d62af7 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.780622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40291bdd-ad4e-40e0-a986-b7aa91d62af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.780886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40291bdd-ad4e-40e0-a986-b7aa91d62af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.791423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40291bdd-ad4e-40e0-a986-b7aa91d62af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.792873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40291bdd-ad4e-40e0-a986-b7aa91d62af7 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.796330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45a3bb-bf3c-4434-82c2-702f2c105b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.796951 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45a3bb-bf3c-4434-82c2-702f2c105b57 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.797229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45a3bb-bf3c-4434-82c2-702f2c105b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.797500 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45a3bb-bf3c-4434-82c2-702f2c105b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-40291bdd-ad4e-40e0-a986-b7aa91d62af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:00.806709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:00.807617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:00.813114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:00.813114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:00.813114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:00.813114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 254/512] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:00.813114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40291bdd-ad4e-40e0-a986-b7aa91d62af7 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.863573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:00.864135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45a3bb-bf3c-4434-82c2-702f2c105b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.865627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45a3bb-bf3c-4434-82c2-702f2c105b57 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.904109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:00.908516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40291bdd-ad4e-40e0-a986-b7aa91d62af7 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:00.940770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb231389-e8f0-4620-a8a5-34bc3ac01f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:00.941314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb231389-e8f0-4620-a8a5-34bc3ac01f01 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:00.941640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb231389-e8f0-4620-a8a5-34bc3ac01f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:00.941911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb231389-e8f0-4620-a8a5-34bc3ac01f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:00.953337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb231389-e8f0-4620-a8a5-34bc3ac01f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:00.954783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb231389-e8f0-4620-a8a5-34bc3ac01f01 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:00.963105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 259/513] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:00.964069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45a3bb-bf3c-4434-82c2-702f2c105b57 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-cb231389-e8f0-4620-a8a5-34bc3ac01f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:00.970039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:00.971120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:00.971956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:00.971956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:00.971956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:00.971956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 255/514] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:00.972175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb231389-e8f0-4620-a8a5-34bc3ac01f01 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.059023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45a3bb-bf3c-4434-82c2-702f2c105b57 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.067805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb231389-e8f0-4620-a8a5-34bc3ac01f01 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.074520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:01.091907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90431f9-d5e9-42a2-a923-0b4244aec26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.092580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90431f9-d5e9-42a2-a923-0b4244aec26d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.092858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90431f9-d5e9-42a2-a923-0b4244aec26d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.093134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90431f9-d5e9-42a2-a923-0b4244aec26d None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.093852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90431f9-d5e9-42a2-a923-0b4244aec26d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:01.094156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90431f9-d5e9-42a2-a923-0b4244aec26d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.095602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90431f9-d5e9-42a2-a923-0b4244aec26d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.101300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ffff9-78da-43d8-95fa-de1870546e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.102426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ffff9-78da-43d8-95fa-de1870546e72 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.102698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ffff9-78da-43d8-95fa-de1870546e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.102974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ffff9-78da-43d8-95fa-de1870546e72 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.103612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ffff9-78da-43d8-95fa-de1870546e72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:01.103914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ffff9-78da-43d8-95fa-de1870546e72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.104030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:01.105250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ffff9-78da-43d8-95fa-de1870546e72 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.114963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 260/515] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:01.115767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90431f9-d5e9-42a2-a923-0b4244aec26d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.121404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 256/516] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:01.121404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ffff9-78da-43d8-95fa-de1870546e72 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.213831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90431f9-d5e9-42a2-a923-0b4244aec26d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.216960 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ffff9-78da-43d8-95fa-de1870546e72 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.246795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10713d89-03a8-49d9-8b65-2bda4fcb4920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.247422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10713d89-03a8-49d9-8b65-2bda4fcb4920 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.247697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10713d89-03a8-49d9-8b65-2bda4fcb4920 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.247963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10713d89-03a8-49d9-8b65-2bda4fcb4920 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.248589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10713d89-03a8-49d9-8b65-2bda4fcb4920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:01.248884 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10713d89-03a8-49d9-8b65-2bda4fcb4920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.250111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532caca7-6d9f-4497-a3ec-432c9bd648fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.250573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10713d89-03a8-49d9-8b65-2bda4fcb4920 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.250733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532caca7-6d9f-4497-a3ec-432c9bd648fc None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.251043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532caca7-6d9f-4497-a3ec-432c9bd648fc None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.251316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532caca7-6d9f-4497-a3ec-432c9bd648fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.261438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 261/517] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:01.261564 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532caca7-6d9f-4497-a3ec-432c9bd648fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.262209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10713d89-03a8-49d9-8b65-2bda4fcb4920 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.263073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532caca7-6d9f-4497-a3ec-432c9bd648fc None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-532caca7-6d9f-4497-a3ec-432c9bd648fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:01.277988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:01.278873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:01.280146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:01.280146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:01.280146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:01.280146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 257/518] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:01.280146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532caca7-6d9f-4497-a3ec-432c9bd648fc None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.298132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:01.326024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:01.369250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10713d89-03a8-49d9-8b65-2bda4fcb4920 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.376192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532caca7-6d9f-4497-a3ec-432c9bd648fc None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.404377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725129af-3dc8-47db-b860-ef8a11734748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.404988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725129af-3dc8-47db-b860-ef8a11734748 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.405263 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725129af-3dc8-47db-b860-ef8a11734748 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.405531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725129af-3dc8-47db-b860-ef8a11734748 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.406169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725129af-3dc8-47db-b860-ef8a11734748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:01.406470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725129af-3dc8-47db-b860-ef8a11734748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.407853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725129af-3dc8-47db-b860-ef8a11734748 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.413136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743adbc-9856-4c29-9dc7-60d5483e5e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.413736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743adbc-9856-4c29-9dc7-60d5483e5e08 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.414041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743adbc-9856-4c29-9dc7-60d5483e5e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.414310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743adbc-9856-4c29-9dc7-60d5483e5e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.419583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 262/519] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:01.420281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725129af-3dc8-47db-b860-ef8a11734748 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.482657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743adbc-9856-4c29-9dc7-60d5483e5e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.484070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743adbc-9856-4c29-9dc7-60d5483e5e08 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.501161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:01.516224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725129af-3dc8-47db-b860-ef8a11734748 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.532619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:01.549952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a164be-ce28-4331-a1c4-fff3fd7ee6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.550550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a164be-ce28-4331-a1c4-fff3fd7ee6f8 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.550829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a164be-ce28-4331-a1c4-fff3fd7ee6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.551137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a164be-ce28-4331-a1c4-fff3fd7ee6f8 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.551775 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a164be-ce28-4331-a1c4-fff3fd7ee6f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.553221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a164be-ce28-4331-a1c4-fff3fd7ee6f8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.574866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 263/520] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:51:01 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:01.575983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a164be-ce28-4331-a1c4-fff3fd7ee6f8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.583856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 258/521] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:01.584560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743adbc-9856-4c29-9dc7-60d5483e5e08 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.696431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743adbc-9856-4c29-9dc7-60d5483e5e08 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.699856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a164be-ce28-4331-a1c4-fff3fd7ee6f8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.715970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:01.736978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f04b1e-8d1d-4787-9acb-406f9296c83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.737295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f04b1e-8d1d-4787-9acb-406f9296c83d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.737512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f04b1e-8d1d-4787-9acb-406f9296c83d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.737781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f04b1e-8d1d-4787-9acb-406f9296c83d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.738123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91830ac1-b6a1-407f-9088-da977208954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.738726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91830ac1-b6a1-407f-9088-da977208954c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.739024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91830ac1-b6a1-407f-9088-da977208954c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.739599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91830ac1-b6a1-407f-9088-da977208954c None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.739957 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91830ac1-b6a1-407f-9088-da977208954c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:01.740253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91830ac1-b6a1-407f-9088-da977208954c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.741607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91830ac1-b6a1-407f-9088-da977208954c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.747138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f04b1e-8d1d-4787-9acb-406f9296c83d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.748589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f04b1e-8d1d-4787-9acb-406f9296c83d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.754879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 259/522] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:01.755772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91830ac1-b6a1-407f-9088-da977208954c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-b3f04b1e-8d1d-4787-9acb-406f9296c83d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:01.761324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:01.762171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:01.763177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:01.763177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:01.763177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:01.763177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:01.763177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:01.763177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:01.763177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:01.763177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:01.763177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 264/523] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:01.763647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f04b1e-8d1d-4787-9acb-406f9296c83d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.801213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:01.866436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91830ac1-b6a1-407f-9088-da977208954c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.880907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f04b1e-8d1d-4787-9acb-406f9296c83d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:01.904927 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c1fed-86db-4fc4-bfb8-a1a465b80427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.906379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c1fed-86db-4fc4-bfb8-a1a465b80427 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.906804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c1fed-86db-4fc4-bfb8-a1a465b80427 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.907405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c1fed-86db-4fc4-bfb8-a1a465b80427 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.915465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfe130-3fb5-4d22-b817-a8d84e42d9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:01.916086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfe130-3fb5-4d22-b817-a8d84e42d9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:01.916360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfe130-3fb5-4d22-b817-a8d84e42d9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:01.916627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfe130-3fb5-4d22-b817-a8d84e42d9b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:01.917170 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c1fed-86db-4fc4-bfb8-a1a465b80427 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.918946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c1fed-86db-4fc4-bfb8-a1a465b80427 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.927274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bfe130-3fb5-4d22-b817-a8d84e42d9b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:01.928715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bfe130-3fb5-4d22-b817-a8d84e42d9b8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-d29c1fed-86db-4fc4-bfb8-a1a465b80427 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:01.933091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:01.934258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 09:51:01.935371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:01.938302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 260/524] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:01.939648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c1fed-86db-4fc4-bfb8-a1a465b80427 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-65bfe130-3fb5-4d22-b817-a8d84e42d9b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:01.944231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:01.945190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:01.946554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:01.946554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:01.946554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:01.950227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 265/525] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:01.951078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfe130-3fb5-4d22-b817-a8d84e42d9b8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:01.988893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:02.041034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:02.058339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfe130-3fb5-4d22-b817-a8d84e42d9b8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:02.058508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c1fed-86db-4fc4-bfb8-a1a465b80427 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:02.095507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17d3d1-aebd-436b-b00d-fc9877bfd60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:02.096118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17d3d1-aebd-436b-b00d-fc9877bfd60f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:02.096399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17d3d1-aebd-436b-b00d-fc9877bfd60f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:02.096558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6942-8c49-4431-b7a6-8922fcf1b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:02.096663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17d3d1-aebd-436b-b00d-fc9877bfd60f None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:02.097147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6942-8c49-4431-b7a6-8922fcf1b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:02.097349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17d3d1-aebd-436b-b00d-fc9877bfd60f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:02.097451 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6942-8c49-4431-b7a6-8922fcf1b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:02.097653 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17d3d1-aebd-436b-b00d-fc9877bfd60f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:02.097744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6942-8c49-4431-b7a6-8922fcf1b7a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:02.099169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17d3d1-aebd-436b-b00d-fc9877bfd60f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:02.105183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb6942-8c49-4431-b7a6-8922fcf1b7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:02.106611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb6942-8c49-4431-b7a6-8922fcf1b7a8 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:02.122911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 266/526] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-31fb6942-8c49-4431-b7a6-8922fcf1b7a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:02.123096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:02.123857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:02.124718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:02.124718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:02.124718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:02.124718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:02.124718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:02.124718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:02.124718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:02.124718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:02.125112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17d3d1-aebd-436b-b00d-fc9877bfd60f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:02.125178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 261/527] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:02.125178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6942-8c49-4431-b7a6-8922fcf1b7a8 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:02.223935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:02.228460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6942-8c49-4431-b7a6-8922fcf1b7a8 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:02.232633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17d3d1-aebd-436b-b00d-fc9877bfd60f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:02.268918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ca49-e48c-4d3c-948b-9732a87aef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:02.269439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ca49-e48c-4d3c-948b-9732a87aef0a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:02.269709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ca49-e48c-4d3c-948b-9732a87aef0a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:02.269979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ca49-e48c-4d3c-948b-9732a87aef0a None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:02.270202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262b9b1-1a51-4f20-8ce7-d7680f0b9a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:02.270613 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ca49-e48c-4d3c-948b-9732a87aef0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:02.270912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ca49-e48c-4d3c-948b-9732a87aef0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:02.271329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b9b1-1a51-4f20-8ce7-d7680f0b9a45 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:02.271768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b9b1-1a51-4f20-8ce7-d7680f0b9a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:02.272165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b9b1-1a51-4f20-8ce7-d7680f0b9a45 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:02.272270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ca49-e48c-4d3c-948b-9732a87aef0a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:02.273225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262b9b1-1a51-4f20-8ce7-d7680f0b9a45 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:02.273675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262b9b1-1a51-4f20-8ce7-d7680f0b9a45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:02.275120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262b9b1-1a51-4f20-8ce7-d7680f0b9a45 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:02.284696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 262/528] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:02.285267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ca49-e48c-4d3c-948b-9732a87aef0a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:02.301926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 267/529] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Aug 10 09:51:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:02.301926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262b9b1-1a51-4f20-8ce7-d7680f0b9a45 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:02.305268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:02.398119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ca49-e48c-4d3c-948b-9732a87aef0a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:02.412878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262b9b1-1a51-4f20-8ce7-d7680f0b9a45 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:02.435254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32c951-8555-4579-ac89-643c16886fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:02.435938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32c951-8555-4579-ac89-643c16886fbf None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:02.436218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32c951-8555-4579-ac89-643c16886fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:02.436486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32c951-8555-4579-ac89-643c16886fbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:02.437197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32c951-8555-4579-ac89-643c16886fbf None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=b8f456f0d15b4d068cfcd6476563247f) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:02.437502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32c951-8555-4579-ac89-643c16886fbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:02.438938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32c951-8555-4579-ac89-643c16886fbf None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:02.444347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 263/530] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:02 2022] GET /identity/v3/role_assignments?user.id=b8f456f0d15b4d068cfcd6476563247f&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:02.445149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32c951-8555-4579-ac89-643c16886fbf None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:02.463347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aab11-ad2f-4b15-8901-64d94e3aa6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:02.464323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aab11-ad2f-4b15-8901-64d94e3aa6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:02.464797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aab11-ad2f-4b15-8901-64d94e3aa6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:02.465288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aab11-ad2f-4b15-8901-64d94e3aa6f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:02.466170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aab11-ad2f-4b15-8901-64d94e3aa6f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:02.466752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aab11-ad2f-4b15-8901-64d94e3aa6f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:02.468355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aab11-ad2f-4b15-8901-64d94e3aa6f9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:02.482130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:02.485607 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 268/531] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:02.486830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aab11-ad2f-4b15-8901-64d94e3aa6f9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:02.562026 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:02.586685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32c951-8555-4579-ac89-643c16886fbf None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:02.619984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aab11-ad2f-4b15-8901-64d94e3aa6f9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:02.640474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd24740-4450-48b9-9cee-718214ca2e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:02.641270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd24740-4450-48b9-9cee-718214ca2e14 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:02.641617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd24740-4450-48b9-9cee-718214ca2e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:02.641958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd24740-4450-48b9-9cee-718214ca2e14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:02.652444 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd24740-4450-48b9-9cee-718214ca2e14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:02.654495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd24740-4450-48b9-9cee-718214ca2e14 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:02.660446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ebec-8b47-43d3-8660-ee2ba5c4e19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:02.661302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ebec-8b47-43d3-8660-ee2ba5c4e19d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:02.661765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ebec-8b47-43d3-8660-ee2ba5c4e19d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:02.662202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ebec-8b47-43d3-8660-ee2ba5c4e19d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-9fd24740-4450-48b9-9cee-718214ca2e14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:02.674798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:02.675695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:02.676520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:02.676520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:02.676520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:02.676520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:02.676520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:02.676520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:02.676520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:02.676520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:02.679029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 264/532] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:02.680577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd24740-4450-48b9-9cee-718214ca2e14 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:02.747214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8ebec-8b47-43d3-8660-ee2ba5c4e19d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:02.748725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8ebec-8b47-43d3-8660-ee2ba5c4e19d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:02.787324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:02.802497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd24740-4450-48b9-9cee-718214ca2e14 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:02.823389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:02.846591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bf1f2-15ba-4b2b-91ce-92b33583d11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:02.847649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bf1f2-15ba-4b2b-91ce-92b33583d11b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:02.847931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bf1f2-15ba-4b2b-91ce-92b33583d11b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:02.848202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bf1f2-15ba-4b2b-91ce-92b33583d11b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:02.848927 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bf1f2-15ba-4b2b-91ce-92b33583d11b None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=904f7f036565419b9fdecc5e47acb45c) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:02.849235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bf1f2-15ba-4b2b-91ce-92b33583d11b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:02.850667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bf1f2-15ba-4b2b-91ce-92b33583d11b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:02.860855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 269/534] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:02.862011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ebec-8b47-43d3-8660-ee2ba5c4e19d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:02.863074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 265/534] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:02 2022] GET /identity/v3/role_assignments?user.id=904f7f036565419b9fdecc5e47acb45c&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:02.866846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bf1f2-15ba-4b2b-91ce-92b33583d11b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:02.982017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ebec-8b47-43d3-8660-ee2ba5c4e19d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:03.023920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640f55a-8fba-4f9f-aa78-a857d4673419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:03.024912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640f55a-8fba-4f9f-aa78-a857d4673419 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:03.025387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640f55a-8fba-4f9f-aa78-a857d4673419 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:03.025848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640f55a-8fba-4f9f-aa78-a857d4673419 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:03.026740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640f55a-8fba-4f9f-aa78-a857d4673419 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:03.027292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640f55a-8fba-4f9f-aa78-a857d4673419 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:03.028919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640f55a-8fba-4f9f-aa78-a857d4673419 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:03.043906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 270/535] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:03.057318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640f55a-8fba-4f9f-aa78-a857d4673419 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:03.081807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bf1f2-15ba-4b2b-91ce-92b33583d11b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:03.108929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:03.147601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd77335-68ce-4e74-8b96-55e66ff63371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:03.148769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd77335-68ce-4e74-8b96-55e66ff63371 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:03.149049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd77335-68ce-4e74-8b96-55e66ff63371 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:03.149323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd77335-68ce-4e74-8b96-55e66ff63371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:03.161769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd77335-68ce-4e74-8b96-55e66ff63371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:03.163247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd77335-68ce-4e74-8b96-55e66ff63371 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:03.180084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-cfd77335-68ce-4e74-8b96-55e66ff63371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:03.194225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:03.195209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:03.196021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:03.196021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:03.196021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:03.196021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 266/536] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:03.201543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd77335-68ce-4e74-8b96-55e66ff63371 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:03.216615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640f55a-8fba-4f9f-aa78-a857d4673419 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:03.271689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ac0c5-5449-45c7-8dc9-9e7a58226bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:03.272302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ac0c5-5449-45c7-8dc9-9e7a58226bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:03.272805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ac0c5-5449-45c7-8dc9-9e7a58226bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:03.272805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ac0c5-5449-45c7-8dc9-9e7a58226bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:03.274085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ac0c5-5449-45c7-8dc9-9e7a58226bf2 None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=f1e3ba8d0c7d4f8190cd0b5b057f5638) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:03.274387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ac0c5-5449-45c7-8dc9-9e7a58226bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:03.275917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ac0c5-5449-45c7-8dc9-9e7a58226bf2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:03.295048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 271/537] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:03 2022] GET /identity/v3/role_assignments?user.id=f1e3ba8d0c7d4f8190cd0b5b057f5638&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:03.366346 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd77335-68ce-4e74-8b96-55e66ff63371 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:03.406499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162fcd0-0599-47c0-9091-da71f0488147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:03.407147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162fcd0-0599-47c0-9091-da71f0488147 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:03.407425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162fcd0-0599-47c0-9091-da71f0488147 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:03.407695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162fcd0-0599-47c0-9091-da71f0488147 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:03.408337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162fcd0-0599-47c0-9091-da71f0488147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:03.408635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162fcd0-0599-47c0-9091-da71f0488147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:03.410021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162fcd0-0599-47c0-9091-da71f0488147 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:03.424055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 267/538] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:03.429800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ac0c5-5449-45c7-8dc9-9e7a58226bf2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:03.553121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:03.569208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ac0c5-5449-45c7-8dc9-9e7a58226bf2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:03.604603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3867f5-b222-4ad3-84ce-bda479847360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:03.605234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3867f5-b222-4ad3-84ce-bda479847360 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:03.605506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3867f5-b222-4ad3-84ce-bda479847360 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:03.605771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3867f5-b222-4ad3-84ce-bda479847360 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:03.612550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3867f5-b222-4ad3-84ce-bda479847360 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:03.614053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3867f5-b222-4ad3-84ce-bda479847360 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6a3867f5-b222-4ad3-84ce-bda479847360 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:03.625789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:03.626610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:03.627735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:03.627735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:03.627735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:03.627735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:03.627735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:03.627735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:03.627735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:03.627735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:03.627735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 272/539] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:03.634687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162fcd0-0599-47c0-9091-da71f0488147 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:03.724145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:03.767176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162fcd0-0599-47c0-9091-da71f0488147 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:03.791729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd87f8a-5a50-4055-920a-3e26ddc8fd42 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:03.792112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd87f8a-5a50-4055-920a-3e26ddc8fd42 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:03.792388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd87f8a-5a50-4055-920a-3e26ddc8fd42 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:03.793949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 273/540] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:03.799178 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e58c-c057-42e9-ba90-4d7e5e04997e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:03.799467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e58c-c057-42e9-ba90-4d7e5e04997e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:03.799734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e58c-c057-42e9-ba90-4d7e5e04997e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:03.803591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce8e2a-e14e-449f-ade6-6b90a0279fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:03.804197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce8e2a-e14e-449f-ade6-6b90a0279fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:03.805084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce8e2a-e14e-449f-ade6-6b90a0279fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:03.805358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce8e2a-e14e-449f-ade6-6b90a0279fd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:03.806113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce8e2a-e14e-449f-ade6-6b90a0279fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:03.806665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce8e2a-e14e-449f-ade6-6b90a0279fd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:03.809025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce8e2a-e14e-449f-ade6-6b90a0279fd2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:03.820843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ea52e58c-c057-42e9-ba90-4d7e5e04997e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:03.822035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ea52e58c-c057-42e9-ba90-4d7e5e04997e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:03.827646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 268/541] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:03.844305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce8e2a-e14e-449f-ade6-6b90a0279fd2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:03.901177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 274/542] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:51:03.935474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e58c-c057-42e9-ba90-4d7e5e04997e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:03.997968 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce8e2a-e14e-449f-ade6-6b90a0279fd2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:04.021923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:04.032021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:04.048411 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e58c-c057-42e9-ba90-4d7e5e04997e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:04.064580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6045f2b-99fe-42d9-a21f-b49b22c0aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:04.065204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6045f2b-99fe-42d9-a21f-b49b22c0aaee None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:04.065484 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6045f2b-99fe-42d9-a21f-b49b22c0aaee None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:04.065800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6045f2b-99fe-42d9-a21f-b49b22c0aaee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:04.066480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6045f2b-99fe-42d9-a21f-b49b22c0aaee None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=ab9b7576a3834862af6abb7e0416f158) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:04.066787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6045f2b-99fe-42d9-a21f-b49b22c0aaee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:04.068263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6045f2b-99fe-42d9-a21f-b49b22c0aaee None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:04.083002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 269/543] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:03 2022] GET /identity/v3/role_assignments?user.id=ab9b7576a3834862af6abb7e0416f158&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:04.104566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5f51b-d625-41cb-a091-eab9b20562fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:04.105692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5f51b-d625-41cb-a091-eab9b20562fc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:04.106171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5f51b-d625-41cb-a091-eab9b20562fc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:04.106638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5f51b-d625-41cb-a091-eab9b20562fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:04.118788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5f51b-d625-41cb-a091-eab9b20562fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-52a5f51b-d625-41cb-a091-eab9b20562fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:04.126762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:04.131663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:04.132456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 275/544] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:04.138732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6045f2b-99fe-42d9-a21f-b49b22c0aaee None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:04.290552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6045f2b-99fe-42d9-a21f-b49b22c0aaee None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:04.335215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dde9e4-39a6-4909-821b-0fb418cc19e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:04.336224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dde9e4-39a6-4909-821b-0fb418cc19e2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:04.336714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dde9e4-39a6-4909-821b-0fb418cc19e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:04.337189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dde9e4-39a6-4909-821b-0fb418cc19e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:04.338032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dde9e4-39a6-4909-821b-0fb418cc19e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:04.338522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dde9e4-39a6-4909-821b-0fb418cc19e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:04.340140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dde9e4-39a6-4909-821b-0fb418cc19e2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:04.354548 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 270/545] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:04.360380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5f51b-d625-41cb-a091-eab9b20562fc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:04.480281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5f51b-d625-41cb-a091-eab9b20562fc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:04.515646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c35ba6-7d35-481a-a5fa-279101bc3b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:04.516282 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c35ba6-7d35-481a-a5fa-279101bc3b17 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:04.516557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c35ba6-7d35-481a-a5fa-279101bc3b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:04.516827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c35ba6-7d35-481a-a5fa-279101bc3b17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:04.519837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:04.528156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c35ba6-7d35-481a-a5fa-279101bc3b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:04.529691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c35ba6-7d35-481a-a5fa-279101bc3b17 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-a2c35ba6-7d35-481a-a5fa-279101bc3b17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:04.545302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:04.546371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:04.547416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:04.547416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:04.547416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:04.549564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 276/546] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:04.555415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dde9e4-39a6-4909-821b-0fb418cc19e2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:04.628937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:04.714392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dde9e4-39a6-4909-821b-0fb418cc19e2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:04.765209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844774f5-e3c2-4599-8991-eecbd96a114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:04.765829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844774f5-e3c2-4599-8991-eecbd96a114c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:04.766111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844774f5-e3c2-4599-8991-eecbd96a114c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:04.766381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844774f5-e3c2-4599-8991-eecbd96a114c None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:04.767052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844774f5-e3c2-4599-8991-eecbd96a114c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:04.767359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844774f5-e3c2-4599-8991-eecbd96a114c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:04.768737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844774f5-e3c2-4599-8991-eecbd96a114c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:04.785920 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 271/547] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:04.792437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c35ba6-7d35-481a-a5fa-279101bc3b17 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:04.839759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893be440-9c13-43ea-b2e7-e7d6b2248a7e None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:04.840051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893be440-9c13-43ea-b2e7-e7d6b2248a7e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:04.840214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893be440-9c13-43ea-b2e7-e7d6b2248a7e None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:04.841783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 272/548] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:04.846653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd6e31-c558-4a06-baf5-e91c77aa1e97 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:04.846931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd6e31-c558-4a06-baf5-e91c77aa1e97 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:04.847231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd6e31-c558-4a06-baf5-e91c77aa1e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:04.865701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0efd6e31-c558-4a06-baf5-e91c77aa1e97 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:04.866826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0efd6e31-c558-4a06-baf5-e91c77aa1e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:04.922508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c35ba6-7d35-481a-a5fa-279101bc3b17 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:04.963735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a719ac-d95f-4dd5-bb5a-e7596aa7e34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:04.964724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a719ac-d95f-4dd5-bb5a-e7596aa7e34b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:04.965215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a719ac-d95f-4dd5-bb5a-e7596aa7e34b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:04.965702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a719ac-d95f-4dd5-bb5a-e7596aa7e34b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:04.975191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a719ac-d95f-4dd5-bb5a-e7596aa7e34b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:04.976910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a719ac-d95f-4dd5-bb5a-e7596aa7e34b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-b9a719ac-d95f-4dd5-bb5a-e7596aa7e34b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:04.989112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:04.990007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:04.991621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:04.994848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 277/549] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:04.999787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a719ac-d95f-4dd5-bb5a-e7596aa7e34b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:05.008493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 273/550] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:51:05.033618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd6e31-c558-4a06-baf5-e91c77aa1e97 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:05.079365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:05.119452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:05.140325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a719ac-d95f-4dd5-bb5a-e7596aa7e34b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:05.210604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d8952-3ccf-4882-840b-5f7f25efa680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:05.211788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d8952-3ccf-4882-840b-5f7f25efa680 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:05.212063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d8952-3ccf-4882-840b-5f7f25efa680 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:05.212332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d8952-3ccf-4882-840b-5f7f25efa680 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:05.213071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d8952-3ccf-4882-840b-5f7f25efa680 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:05.213378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d8952-3ccf-4882-840b-5f7f25efa680 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:05.216918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d8952-3ccf-4882-840b-5f7f25efa680 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:05.246387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd6e31-c558-4a06-baf5-e91c77aa1e97 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:05.255238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 278/551] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 10 09:51:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:05.257463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d8952-3ccf-4882-840b-5f7f25efa680 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:05.312735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e032322-b0aa-42bc-9942-14537162eafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:05.313364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e032322-b0aa-42bc-9942-14537162eafb None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:05.313635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e032322-b0aa-42bc-9942-14537162eafb None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:05.313898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e032322-b0aa-42bc-9942-14537162eafb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:05.335986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e032322-b0aa-42bc-9942-14537162eafb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-0e032322-b0aa-42bc-9942-14537162eafb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:05.340857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:05.341740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:05.342700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:05.343887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 274/552] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:05.349008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e032322-b0aa-42bc-9942-14537162eafb None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:05.423278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d8952-3ccf-4882-840b-5f7f25efa680 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:05.433393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:05.466210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b3d1b-7e16-40de-a791-7808cca5f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:05.467007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b3d1b-7e16-40de-a791-7808cca5f66d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:05.467392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b3d1b-7e16-40de-a791-7808cca5f66d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:05.467755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b3d1b-7e16-40de-a791-7808cca5f66d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:05.547576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b3d1b-7e16-40de-a791-7808cca5f66d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:05.549244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b3d1b-7e16-40de-a791-7808cca5f66d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:05.552905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:05.568124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e032322-b0aa-42bc-9942-14537162eafb None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:05.629698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cb24a-e8e7-4612-8261-0e431e61fc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:05.630298 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cb24a-e8e7-4612-8261-0e431e61fc22 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:05.630572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cb24a-e8e7-4612-8261-0e431e61fc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:05.632106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cb24a-e8e7-4612-8261-0e431e61fc22 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:05.632805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cb24a-e8e7-4612-8261-0e431e61fc22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:05.633106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cb24a-e8e7-4612-8261-0e431e61fc22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:05.635367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cb24a-e8e7-4612-8261-0e431e61fc22 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:05.650275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 275/553] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:05.655876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cb24a-e8e7-4612-8261-0e431e61fc22 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:05.722537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 279/554] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 09:51:05.724715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e953ff-cb48-4df3-a4a6-70d40ac7f9e7 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:05.725205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e953ff-cb48-4df3-a4a6-70d40ac7f9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:05.725672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e953ff-cb48-4df3-a4a6-70d40ac7f9e7 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:05.731508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 280/555] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:05.732428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e953ff-cb48-4df3-a4a6-70d40ac7f9e7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:05.744696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:05.804753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cb24a-e8e7-4612-8261-0e431e61fc22 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:05.840304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460d1ea-8d2e-40bb-b432-5a60730fdc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:05.840929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460d1ea-8d2e-40bb-b432-5a60730fdc5a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:05.841211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460d1ea-8d2e-40bb-b432-5a60730fdc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:05.841480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460d1ea-8d2e-40bb-b432-5a60730fdc5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:05.851991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460d1ea-8d2e-40bb-b432-5a60730fdc5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:05.853431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460d1ea-8d2e-40bb-b432-5a60730fdc5a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:05.860808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e953ff-cb48-4df3-a4a6-70d40ac7f9e7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-9460d1ea-8d2e-40bb-b432-5a60730fdc5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:05.867240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:05.868135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:05.869273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:05.869273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:05.869273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:05.871273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 276/556] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:05.873368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d98d0-b226-46f9-a085-18fb834244e4 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:05.873651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d98d0-b226-46f9-a085-18fb834244e4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:05.874320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d98d0-b226-46f9-a085-18fb834244e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:05.892590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-bd8d98d0-b226-46f9-a085-18fb834244e4 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:05.893755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bd8d98d0-b226-46f9-a085-18fb834244e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:05.895746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:05.903079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582131d5-cc6e-4bec-838f-052c803ec030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:05.903712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582131d5-cc6e-4bec-838f-052c803ec030 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:05.903989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582131d5-cc6e-4bec-838f-052c803ec030 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:05.904262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582131d5-cc6e-4bec-838f-052c803ec030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:05.914287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582131d5-cc6e-4bec-838f-052c803ec030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:05.915759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582131d5-cc6e-4bec-838f-052c803ec030 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-582131d5-cc6e-4bec-838f-052c803ec030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:05.930691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:05.932272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:05.933392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:05.933392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:05.933392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:05.933392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 281/557] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:05.933392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e893-04b7-433b-b1e8-3f68023c8e5b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:05.933392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e893-04b7-433b-b1e8-3f68023c8e5b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:05.933722 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e893-04b7-433b-b1e8-3f68023c8e5b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:05.934842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 282/558] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:05.939310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e893-04b7-433b-b1e8-3f68023c8e5b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:05.974778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 277/559] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:05.975743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d98d0-b226-46f9-a085-18fb834244e4 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.004091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:06.049136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e893-04b7-433b-b1e8-3f68023c8e5b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.089060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75024a-b31d-4c70-8c95-aac20f932292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.089668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75024a-b31d-4c70-8c95-aac20f932292 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.090001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75024a-b31d-4c70-8c95-aac20f932292 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.090234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d98d0-b226-46f9-a085-18fb834244e4 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.090336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75024a-b31d-4c70-8c95-aac20f932292 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.090905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75024a-b31d-4c70-8c95-aac20f932292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:06.091231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75024a-b31d-4c70-8c95-aac20f932292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:06.093688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75024a-b31d-4c70-8c95-aac20f932292 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:06.106980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 283/560] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:06.108291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe461b-21df-44cf-9f20-ec69a6a832dc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.108574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe461b-21df-44cf-9f20-ec69a6a832dc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.108845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe461b-21df-44cf-9f20-ec69a6a832dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.127405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b4fe461b-21df-44cf-9f20-ec69a6a832dc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:06.128601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b4fe461b-21df-44cf-9f20-ec69a6a832dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:06.132440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e87a6f-5a2d-4c52-bc56-d58cd1d57f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.133813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e87a6f-5a2d-4c52-bc56-d58cd1d57f5b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.134365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e87a6f-5a2d-4c52-bc56-d58cd1d57f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.134778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e87a6f-5a2d-4c52-bc56-d58cd1d57f5b None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.135624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e87a6f-5a2d-4c52-bc56-d58cd1d57f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:06.136113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e87a6f-5a2d-4c52-bc56-d58cd1d57f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:06.138097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e87a6f-5a2d-4c52-bc56-d58cd1d57f5b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:06.147689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:06.158996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 278/561] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:06.158996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e87a6f-5a2d-4c52-bc56-d58cd1d57f5b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.203362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 284/562] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:06.204553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe461b-21df-44cf-9f20-ec69a6a832dc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.262748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:06.281111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e87a6f-5a2d-4c52-bc56-d58cd1d57f5b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.315561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe461b-21df-44cf-9f20-ec69a6a832dc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.318746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1698842-699a-468b-a1cb-468c4034a771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.319595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1698842-699a-468b-a1cb-468c4034a771 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.320640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1698842-699a-468b-a1cb-468c4034a771 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.321155 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1698842-699a-468b-a1cb-468c4034a771 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.332220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1698842-699a-468b-a1cb-468c4034a771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-b1698842-699a-468b-a1cb-468c4034a771 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:06.337711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:06.338653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:06.339517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:06.341414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 279/563] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:06.342440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1698842-699a-468b-a1cb-468c4034a771 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.355038 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260394a-b9af-4461-8370-69012e768a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.355667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260394a-b9af-4461-8370-69012e768a86 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.355944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260394a-b9af-4461-8370-69012e768a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.356213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260394a-b9af-4461-8370-69012e768a86 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.363674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260394a-b9af-4461-8370-69012e768a86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:06.365128 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260394a-b9af-4461-8370-69012e768a86 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f260394a-b9af-4461-8370-69012e768a86 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:06.377553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:06.378529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 285/564] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:06.379494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260394a-b9af-4461-8370-69012e768a86 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.417029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:06.459513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1698842-699a-468b-a1cb-468c4034a771 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.483862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260394a-b9af-4461-8370-69012e768a86 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.491807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973470b-6bbf-4039-b19e-96f319dd8a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.493007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973470b-6bbf-4039-b19e-96f319dd8a6b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.493558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973470b-6bbf-4039-b19e-96f319dd8a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.493832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973470b-6bbf-4039-b19e-96f319dd8a6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.493961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:06.500846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973470b-6bbf-4039-b19e-96f319dd8a6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:06.502284 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973470b-6bbf-4039-b19e-96f319dd8a6b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-b973470b-6bbf-4039-b19e-96f319dd8a6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:06.513179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:06.514191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 280/565] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:06.515254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973470b-6bbf-4039-b19e-96f319dd8a6b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.521731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d67d82-5ca7-4f2a-b73f-dc5aacc0851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.522361 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d67d82-5ca7-4f2a-b73f-dc5aacc0851f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.522640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d67d82-5ca7-4f2a-b73f-dc5aacc0851f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.522912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d67d82-5ca7-4f2a-b73f-dc5aacc0851f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.533153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d67d82-5ca7-4f2a-b73f-dc5aacc0851f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-87d67d82-5ca7-4f2a-b73f-dc5aacc0851f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:06.538279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:06.539890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 286/566] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:06.540887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d67d82-5ca7-4f2a-b73f-dc5aacc0851f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.618752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973470b-6bbf-4039-b19e-96f319dd8a6b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.634905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:06.639530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d67d82-5ca7-4f2a-b73f-dc5aacc0851f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.652010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6b24c-e1ad-41d7-b86b-2d038499491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.652620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6b24c-e1ad-41d7-b86b-2d038499491a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.652905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6b24c-e1ad-41d7-b86b-2d038499491a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.653176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6b24c-e1ad-41d7-b86b-2d038499491a None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.653830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6b24c-e1ad-41d7-b86b-2d038499491a None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:06.654131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6b24c-e1ad-41d7-b86b-2d038499491a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:06.655537 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6b24c-e1ad-41d7-b86b-2d038499491a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:06.663109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 281/567] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:06.664772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8aec0-a181-49c6-bb6a-ac704116bcfc None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.665051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8aec0-a181-49c6-bb6a-ac704116bcfc None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.665323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8aec0-a181-49c6-bb6a-ac704116bcfc None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.666496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 282/568] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:06.667208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8aec0-a181-49c6-bb6a-ac704116bcfc None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.678573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e712ad-f7ed-49ba-a3b9-a5b94467da6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.679211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e712ad-f7ed-49ba-a3b9-a5b94467da6f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.680037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e712ad-f7ed-49ba-a3b9-a5b94467da6f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.680308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e712ad-f7ed-49ba-a3b9-a5b94467da6f None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.681030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e712ad-f7ed-49ba-a3b9-a5b94467da6f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:06.681332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e712ad-f7ed-49ba-a3b9-a5b94467da6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:06.682759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e712ad-f7ed-49ba-a3b9-a5b94467da6f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:06.699371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 287/569] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:06.700174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e712ad-f7ed-49ba-a3b9-a5b94467da6f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.727533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:06.772890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8aec0-a181-49c6-bb6a-ac704116bcfc None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.798647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e712ad-f7ed-49ba-a3b9-a5b94467da6f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.805519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a81f3d-b25c-445c-b10c-5ea07218e7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.806113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a81f3d-b25c-445c-b10c-5ea07218e7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.806390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a81f3d-b25c-445c-b10c-5ea07218e7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.806659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a81f3d-b25c-445c-b10c-5ea07218e7e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.807367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a81f3d-b25c-445c-b10c-5ea07218e7e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:06.807670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a81f3d-b25c-445c-b10c-5ea07218e7e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:06.809086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a81f3d-b25c-445c-b10c-5ea07218e7e2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:06.820643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 283/570] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:06.821495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a81f3d-b25c-445c-b10c-5ea07218e7e2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.833210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b5ab2-94a0-4e57-b667-4fa6da8aecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.833821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b5ab2-94a0-4e57-b667-4fa6da8aecce None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.834100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b5ab2-94a0-4e57-b667-4fa6da8aecce None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.834369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b5ab2-94a0-4e57-b667-4fa6da8aecce None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.835059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b5ab2-94a0-4e57-b667-4fa6da8aecce None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:06.835362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b5ab2-94a0-4e57-b667-4fa6da8aecce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:06.837213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b5ab2-94a0-4e57-b667-4fa6da8aecce None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:06.845074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 288/571] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:06.846198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94aec0-80c6-4610-9ede-85319dcaafc2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.846473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94aec0-80c6-4610-9ede-85319dcaafc2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.846751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94aec0-80c6-4610-9ede-85319dcaafc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.864021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ef94aec0-80c6-4610-9ede-85319dcaafc2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:06.864240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:06.865192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ef94aec0-80c6-4610-9ede-85319dcaafc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:06.921238 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a81f3d-b25c-445c-b10c-5ea07218e7e2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:06.930347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 289/572] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:06.930963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94aec0-80c6-4610-9ede-85319dcaafc2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:06.956414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496fbca-91a5-44fb-a405-79d55c3731f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:06.957034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496fbca-91a5-44fb-a405-79d55c3731f6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:06.957312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496fbca-91a5-44fb-a405-79d55c3731f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:06.957581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496fbca-91a5-44fb-a405-79d55c3731f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:06.958633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:06.969086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496fbca-91a5-44fb-a405-79d55c3731f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:06.970321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496fbca-91a5-44fb-a405-79d55c3731f6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-d496fbca-91a5-44fb-a405-79d55c3731f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:06.984919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:06.985990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:06.986868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:06.986868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:06.986868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:06.986868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 284/573] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:06.987116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496fbca-91a5-44fb-a405-79d55c3731f6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.042192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94aec0-80c6-4610-9ede-85319dcaafc2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.075998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fd367-9822-4769-982f-3f9319555a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.076625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fd367-9822-4769-982f-3f9319555a31 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.076822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:07.076956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fd367-9822-4769-982f-3f9319555a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.077158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fd367-9822-4769-982f-3f9319555a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.085778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496fbca-91a5-44fb-a405-79d55c3731f6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.086906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fd367-9822-4769-982f-3f9319555a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.088361 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fd367-9822-4769-982f-3f9319555a31 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-5a8fd367-9822-4769-982f-3f9319555a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:07.102079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:07.103205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:07.104075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:07.104075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:07.104075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:07.104075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 290/574] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:07.104075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fd367-9822-4769-982f-3f9319555a31 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.119114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4f394-e244-46ac-932c-efa4e4346b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.119710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4f394-e244-46ac-932c-efa4e4346b23 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.119986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4f394-e244-46ac-932c-efa4e4346b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.120255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4f394-e244-46ac-932c-efa4e4346b23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.120979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4f394-e244-46ac-932c-efa4e4346b23 None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=af29f3451f654f53ac98239a2b28d128) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:07.121285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4f394-e244-46ac-932c-efa4e4346b23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.122698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4f394-e244-46ac-932c-efa4e4346b23 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.129324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 285/575] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:06 2022] GET /identity/v3/role_assignments?user.id=af29f3451f654f53ac98239a2b28d128&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:07.130445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4f394-e244-46ac-932c-efa4e4346b23 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.180669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:07.217727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fd367-9822-4769-982f-3f9319555a31 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.251233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4f394-e244-46ac-932c-efa4e4346b23 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.257250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4efbc2-198f-4882-8f99-9f12f7d57e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.258424 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4efbc2-198f-4882-8f99-9f12f7d57e21 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.258703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4efbc2-198f-4882-8f99-9f12f7d57e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.259608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4efbc2-198f-4882-8f99-9f12f7d57e21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.271963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4efbc2-198f-4882-8f99-9f12f7d57e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.274471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4efbc2-198f-4882-8f99-9f12f7d57e21 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.289254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab4f49-d56e-4497-8736-9cbadca7afbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.289884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab4f49-d56e-4497-8736-9cbadca7afbe None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-3e4efbc2-198f-4882-8f99-9f12f7d57e21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:07.290118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:07.291216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:07.292320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:07.292320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:07.292320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:07.292546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab4f49-d56e-4497-8736-9cbadca7afbe None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.292546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab4f49-d56e-4497-8736-9cbadca7afbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.292704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 291/576] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:07.292704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4efbc2-198f-4882-8f99-9f12f7d57e21 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.306087 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab4f49-d56e-4497-8736-9cbadca7afbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-e7ab4f49-d56e-4497-8736-9cbadca7afbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:07.311732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:07.312808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:07.313837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:07.317798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 286/577] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:07.318624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab4f49-d56e-4497-8736-9cbadca7afbe None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.337573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:07.394336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4efbc2-198f-4882-8f99-9f12f7d57e21 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.423703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:07.430193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab4f49-d56e-4497-8736-9cbadca7afbe None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.437121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81215df-8707-439f-9060-3293af14f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.437732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81215df-8707-439f-9060-3293af14f452 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.438013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81215df-8707-439f-9060-3293af14f452 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.438286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81215df-8707-439f-9060-3293af14f452 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.439030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81215df-8707-439f-9060-3293af14f452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:07.439343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81215df-8707-439f-9060-3293af14f452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.440705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81215df-8707-439f-9060-3293af14f452 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.457822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 292/578] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:07.458747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81215df-8707-439f-9060-3293af14f452 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.465169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc55f5-eac4-4be3-a431-cc2aa9d68501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.465771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc55f5-eac4-4be3-a431-cc2aa9d68501 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.466047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc55f5-eac4-4be3-a431-cc2aa9d68501 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.466315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc55f5-eac4-4be3-a431-cc2aa9d68501 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.466969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc55f5-eac4-4be3-a431-cc2aa9d68501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:07.467274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc55f5-eac4-4be3-a431-cc2aa9d68501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.468622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc55f5-eac4-4be3-a431-cc2aa9d68501 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.484626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 287/579] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:07.484626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc55f5-eac4-4be3-a431-cc2aa9d68501 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.568962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:07.570865 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81215df-8707-439f-9060-3293af14f452 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.597575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc55f5-eac4-4be3-a431-cc2aa9d68501 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.605228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5960d-594a-418d-9648-963a576c7699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.606058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5960d-594a-418d-9648-963a576c7699 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.606342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5960d-594a-418d-9648-963a576c7699 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.606618 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5960d-594a-418d-9648-963a576c7699 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.607311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5960d-594a-418d-9648-963a576c7699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:07.607628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5960d-594a-418d-9648-963a576c7699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.608962 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5960d-594a-418d-9648-963a576c7699 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.621728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 293/580] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:07.622532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5960d-594a-418d-9648-963a576c7699 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.638615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9c25e-0f2a-4b77-8a9a-bb63f9be562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.639253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9c25e-0f2a-4b77-8a9a-bb63f9be562c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.639823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9c25e-0f2a-4b77-8a9a-bb63f9be562c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.639977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9c25e-0f2a-4b77-8a9a-bb63f9be562c None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.640624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9c25e-0f2a-4b77-8a9a-bb63f9be562c None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:07.640931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9c25e-0f2a-4b77-8a9a-bb63f9be562c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.642304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9c25e-0f2a-4b77-8a9a-bb63f9be562c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.651282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 288/581] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:07.652266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9c25e-0f2a-4b77-8a9a-bb63f9be562c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.669296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:07.729392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5960d-594a-418d-9648-963a576c7699 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.764252 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9c25e-0f2a-4b77-8a9a-bb63f9be562c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.766004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d358c-1a07-4845-93a9-02f7d1548704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.766637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d358c-1a07-4845-93a9-02f7d1548704 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.766911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d358c-1a07-4845-93a9-02f7d1548704 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.767209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d358c-1a07-4845-93a9-02f7d1548704 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.774730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d358c-1a07-4845-93a9-02f7d1548704 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.776227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d358c-1a07-4845-93a9-02f7d1548704 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-030d358c-1a07-4845-93a9-02f7d1548704 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:07.787347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:07.788275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 294/582] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:07.789266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d358c-1a07-4845-93a9-02f7d1548704 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.805798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3209cb-5b44-45ea-835a-9b5463e75b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.806521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3209cb-5b44-45ea-835a-9b5463e75b9e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.806799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3209cb-5b44-45ea-835a-9b5463e75b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.807097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3209cb-5b44-45ea-835a-9b5463e75b9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.814505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3209cb-5b44-45ea-835a-9b5463e75b9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.815958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3209cb-5b44-45ea-835a-9b5463e75b9e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.817771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-2e3209cb-5b44-45ea-835a-9b5463e75b9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:07.829436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:07.830307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:07.831254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:07.831254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:07.831254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:07.831254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:07.831254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:07.831254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:07.831254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:07.831254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:07.833278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 289/583] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:07.834253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3209cb-5b44-45ea-835a-9b5463e75b9e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.899180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d358c-1a07-4845-93a9-02f7d1548704 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.900996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:07.936690 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5571de6-aea3-4896-9973-c3693eba61b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.937308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5571de6-aea3-4896-9973-c3693eba61b2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.937585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5571de6-aea3-4896-9973-c3693eba61b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.937852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5571de6-aea3-4896-9973-c3693eba61b2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.942749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3209cb-5b44-45ea-835a-9b5463e75b9e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:07.944817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5571de6-aea3-4896-9973-c3693eba61b2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.946847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5571de6-aea3-4896-9973-c3693eba61b2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-a5571de6-aea3-4896-9973-c3693eba61b2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:07.958014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:07.958938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 295/584] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:07.959968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5571de6-aea3-4896-9973-c3693eba61b2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:07.981385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16d149-aaed-48f6-9cbf-7244fc6f266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:07.982010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16d149-aaed-48f6-9cbf-7244fc6f266a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:07.982292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16d149-aaed-48f6-9cbf-7244fc6f266a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:07.982574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16d149-aaed-48f6-9cbf-7244fc6f266a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:07.994014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16d149-aaed-48f6-9cbf-7244fc6f266a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:07.995477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16d149-aaed-48f6-9cbf-7244fc6f266a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-fe16d149-aaed-48f6-9cbf-7244fc6f266a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:08.009431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:08.010547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:08.011853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:08.011853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:08.011853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:08.015129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 290/585] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:08.015949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16d149-aaed-48f6-9cbf-7244fc6f266a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.066679 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:08.068512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5571de6-aea3-4896-9973-c3693eba61b2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.104406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989f49c-4b6c-45ac-844b-f78a4f3bce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.105033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989f49c-4b6c-45ac-844b-f78a4f3bce64 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.105376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989f49c-4b6c-45ac-844b-f78a4f3bce64 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.105726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989f49c-4b6c-45ac-844b-f78a4f3bce64 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.106528 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989f49c-4b6c-45ac-844b-f78a4f3bce64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:08.106840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989f49c-4b6c-45ac-844b-f78a4f3bce64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.108294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989f49c-4b6c-45ac-844b-f78a4f3bce64 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.122611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16d149-aaed-48f6-9cbf-7244fc6f266a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.126368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 296/586] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 10 09:51:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:08.127480 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989f49c-4b6c-45ac-844b-f78a4f3bce64 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.150636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:08.167148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548cd7b-e87f-4da0-86f8-6f40b70bc62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.167961 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548cd7b-e87f-4da0-86f8-6f40b70bc62e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.168385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548cd7b-e87f-4da0-86f8-6f40b70bc62e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.168800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548cd7b-e87f-4da0-86f8-6f40b70bc62e None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.169633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548cd7b-e87f-4da0-86f8-6f40b70bc62e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:08.170106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548cd7b-e87f-4da0-86f8-6f40b70bc62e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.171763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548cd7b-e87f-4da0-86f8-6f40b70bc62e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.185553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 291/587] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:08.186845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548cd7b-e87f-4da0-86f8-6f40b70bc62e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.232587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989f49c-4b6c-45ac-844b-f78a4f3bce64 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.269587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b7780-9322-438d-9042-0c59e297fb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.270195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b7780-9322-438d-9042-0c59e297fb17 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.270476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b7780-9322-438d-9042-0c59e297fb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.270746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b7780-9322-438d-9042-0c59e297fb17 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.271506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b7780-9322-438d-9042-0c59e297fb17 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:08.271811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b7780-9322-438d-9042-0c59e297fb17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.273226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b7780-9322-438d-9042-0c59e297fb17 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.290776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 297/588] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:08.291629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b7780-9322-438d-9042-0c59e297fb17 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.305674 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548cd7b-e87f-4da0-86f8-6f40b70bc62e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.316556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:08.342554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1e812-9f56-4139-9ce2-f227a0223f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.343274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1e812-9f56-4139-9ce2-f227a0223f32 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.343548 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1e812-9f56-4139-9ce2-f227a0223f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.343824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1e812-9f56-4139-9ce2-f227a0223f32 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.344431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1e812-9f56-4139-9ce2-f227a0223f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:08.344725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1e812-9f56-4139-9ce2-f227a0223f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.346055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1e812-9f56-4139-9ce2-f227a0223f32 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.358559 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 292/589] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:08.359339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1e812-9f56-4139-9ce2-f227a0223f32 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.378569 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:08.405836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b7780-9322-438d-9042-0c59e297fb17 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.448023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f6bd3-39f5-4b12-b73d-9cc80b3150ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.448740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f6bd3-39f5-4b12-b73d-9cc80b3150ab None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.448924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f6bd3-39f5-4b12-b73d-9cc80b3150ab None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.449193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f6bd3-39f5-4b12-b73d-9cc80b3150ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.459748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f6bd3-39f5-4b12-b73d-9cc80b3150ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.461723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f6bd3-39f5-4b12-b73d-9cc80b3150ab None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.467186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1e812-9f56-4139-9ce2-f227a0223f32 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ab5f6bd3-39f5-4b12-b73d-9cc80b3150ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:08.478141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:08.479117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:08.480050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:08.480050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:08.480050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:08.480050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 298/590] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:08.480273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f6bd3-39f5-4b12-b73d-9cc80b3150ab None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.508145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d029d-0158-40b6-bba9-382f9ad15ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.508795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d029d-0158-40b6-bba9-382f9ad15ee4 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.509070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d029d-0158-40b6-bba9-382f9ad15ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.509342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d029d-0158-40b6-bba9-382f9ad15ee4 None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/b8f456f0d15b4d068cfcd6476563247f/roles/bacea3b4b26e48a598b75ea1b301fafb` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.524384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d029d-0158-40b6-bba9-382f9ad15ee4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=b8f456f0d15b4d068cfcd6476563247f, role_id=bacea3b4b26e48a598b75ea1b301fafb)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.525817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d029d-0158-40b6-bba9-382f9ad15ee4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.540083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 293/591] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:51:08 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/b8f456f0d15b4d068cfcd6476563247f/roles/bacea3b4b26e48a598b75ea1b301fafb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:51:08.541250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d029d-0158-40b6-bba9-382f9ad15ee4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.570261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:08.607842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f6bd3-39f5-4b12-b73d-9cc80b3150ab None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.646440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:08.653919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be16e13-3a17-426f-875b-091f99b0c40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.654918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be16e13-3a17-426f-875b-091f99b0c40e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.655451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be16e13-3a17-426f-875b-091f99b0c40e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.655896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be16e13-3a17-426f-875b-091f99b0c40e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.668866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be16e13-3a17-426f-875b-091f99b0c40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.670695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be16e13-3a17-426f-875b-091f99b0c40e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.676489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d029d-0158-40b6-bba9-382f9ad15ee4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0be16e13-3a17-426f-875b-091f99b0c40e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:08.688926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:08.689925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:08.690793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:08.690793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:08.690793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:08.699437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 299/592] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:08.700711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be16e13-3a17-426f-875b-091f99b0c40e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.727811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af2741-890f-4dad-b374-5c656953c27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.728459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af2741-890f-4dad-b374-5c656953c27d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.728738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af2741-890f-4dad-b374-5c656953c27d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.729011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af2741-890f-4dad-b374-5c656953c27d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.738684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af2741-890f-4dad-b374-5c656953c27d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.740432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af2741-890f-4dad-b374-5c656953c27d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-15af2741-890f-4dad-b374-5c656953c27d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:08.757103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:08.758598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:08.759580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 294/593] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:08.760172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af2741-890f-4dad-b374-5c656953c27d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.832118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be16e13-3a17-426f-875b-091f99b0c40e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.850246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:08.873158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af2741-890f-4dad-b374-5c656953c27d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:08.878104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ed403-355c-4e9c-8a55-4af8b53cbd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.878935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ed403-355c-4e9c-8a55-4af8b53cbd63 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.879390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ed403-355c-4e9c-8a55-4af8b53cbd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.879803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ed403-355c-4e9c-8a55-4af8b53cbd63 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.880579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ed403-355c-4e9c-8a55-4af8b53cbd63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:08.881041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ed403-355c-4e9c-8a55-4af8b53cbd63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.883086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ed403-355c-4e9c-8a55-4af8b53cbd63 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.895641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 300/594] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:08.901771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ed403-355c-4e9c-8a55-4af8b53cbd63 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.909632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ae5fe-a1fd-4bb5-b028-48ff838d00dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:08.910274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ae5fe-a1fd-4bb5-b028-48ff838d00dc None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:08.910555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ae5fe-a1fd-4bb5-b028-48ff838d00dc None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:08.910882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ae5fe-a1fd-4bb5-b028-48ff838d00dc None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:08.911637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ae5fe-a1fd-4bb5-b028-48ff838d00dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:08.911947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ae5fe-a1fd-4bb5-b028-48ff838d00dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:08.913318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ae5fe-a1fd-4bb5-b028-48ff838d00dc None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:08.930754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 295/595] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:08.930754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ae5fe-a1fd-4bb5-b028-48ff838d00dc None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:08.959012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:09.038588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ed403-355c-4e9c-8a55-4af8b53cbd63 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.039862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ae5fe-a1fd-4bb5-b028-48ff838d00dc None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.076073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b978daf-ca53-432a-b3e7-16480e5b98da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.076991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b978daf-ca53-432a-b3e7-16480e5b98da None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.077342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b978daf-ca53-432a-b3e7-16480e5b98da None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.077623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b978daf-ca53-432a-b3e7-16480e5b98da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.082790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11a4e2-1df5-4e0b-bd01-ada0e77f6e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.083782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11a4e2-1df5-4e0b-bd01-ada0e77f6e5a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.084317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11a4e2-1df5-4e0b-bd01-ada0e77f6e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.084703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b978daf-ca53-432a-b3e7-16480e5b98da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:09.084888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11a4e2-1df5-4e0b-bd01-ada0e77f6e5a None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.085794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11a4e2-1df5-4e0b-bd01-ada0e77f6e5a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:09.086262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11a4e2-1df5-4e0b-bd01-ada0e77f6e5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:09.086438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b978daf-ca53-432a-b3e7-16480e5b98da None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:09.088456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11a4e2-1df5-4e0b-bd01-ada0e77f6e5a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:09.089753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-4b978daf-ca53-432a-b3e7-16480e5b98da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:09.099775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:09.100864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:09.101977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:09.101977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:09.101977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:09.101977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:09.101977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:09.101977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:09.101977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:09.101977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:09.107001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 296/596] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:09.107001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b978daf-ca53-432a-b3e7-16480e5b98da None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:09.120993 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 301/597] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Aug 10 09:51:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:09.122213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11a4e2-1df5-4e0b-bd01-ada0e77f6e5a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:09.219615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b978daf-ca53-432a-b3e7-16480e5b98da None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.222993 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:09.228096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11a4e2-1df5-4e0b-bd01-ada0e77f6e5a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.258459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405037c2-685c-4cce-99cc-3b4c47ca9d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.259568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405037c2-685c-4cce-99cc-3b4c47ca9d4a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.260049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405037c2-685c-4cce-99cc-3b4c47ca9d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.260772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405037c2-685c-4cce-99cc-3b4c47ca9d4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.264907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b96d34-6560-4cca-b345-fc490431e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.265517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b96d34-6560-4cca-b345-fc490431e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.265799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b96d34-6560-4cca-b345-fc490431e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.266073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b96d34-6560-4cca-b345-fc490431e1c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.266757 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b96d34-6560-4cca-b345-fc490431e1c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:09.267161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b96d34-6560-4cca-b345-fc490431e1c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:09.268567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b96d34-6560-4cca-b345-fc490431e1c3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:09.269362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405037c2-685c-4cce-99cc-3b4c47ca9d4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:09.271514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405037c2-685c-4cce-99cc-3b4c47ca9d4a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:09.280171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 302/598] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:09.280983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b96d34-6560-4cca-b345-fc490431e1c3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-405037c2-685c-4cce-99cc-3b4c47ca9d4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:09.284464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:09.285516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:09.286614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:09.286614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:09.286614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:09.286614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:09.286614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:09.286614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:09.286614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:09.286614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:09.291032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 297/599] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:09.293145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766408cc-95a2-411c-a7f1-336a95635b98 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.293430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766408cc-95a2-411c-a7f1-336a95635b98 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.293706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766408cc-95a2-411c-a7f1-336a95635b98 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.298929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 298/600] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:09.298929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766408cc-95a2-411c-a7f1-336a95635b98 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:09.365130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:09.392038 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b96d34-6560-4cca-b345-fc490431e1c3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.415547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766408cc-95a2-411c-a7f1-336a95635b98 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.430796 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68260996-c758-4434-b41e-2b2ce4838efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.431453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68260996-c758-4434-b41e-2b2ce4838efe None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.431730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68260996-c758-4434-b41e-2b2ce4838efe None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.431988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68260996-c758-4434-b41e-2b2ce4838efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.443271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68260996-c758-4434-b41e-2b2ce4838efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:09.444869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68260996-c758-4434-b41e-2b2ce4838efe None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:09.457660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494245a-a76d-42c9-ad9d-fdc2dd8dccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.458301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494245a-a76d-42c9-ad9d-fdc2dd8dccd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.458575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494245a-a76d-42c9-ad9d-fdc2dd8dccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.458843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494245a-a76d-42c9-ad9d-fdc2dd8dccd3 None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/904f7f036565419b9fdecc5e47acb45c/roles/bacea3b4b26e48a598b75ea1b301fafb` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-68260996-c758-4434-b41e-2b2ce4838efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:09.460242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:09.461377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:09.462455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:09.462455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:09.462455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:09.463682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 303/601] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:09.464398 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68260996-c758-4434-b41e-2b2ce4838efe None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:09.481097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494245a-a76d-42c9-ad9d-fdc2dd8dccd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=904f7f036565419b9fdecc5e47acb45c, role_id=bacea3b4b26e48a598b75ea1b301fafb)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:09.481282 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:09.483019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494245a-a76d-42c9-ad9d-fdc2dd8dccd3 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:09.502989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 299/602] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:51:09 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/904f7f036565419b9fdecc5e47acb45c/roles/bacea3b4b26e48a598b75ea1b301fafb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:51:09.502989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07df91c-f81d-4885-9f8d-aa2c186f01dc None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.502989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07df91c-f81d-4885-9f8d-aa2c186f01dc None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.502989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07df91c-f81d-4885-9f8d-aa2c186f01dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.526042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d07df91c-f81d-4885-9f8d-aa2c186f01dc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:09.527518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d07df91c-f81d-4885-9f8d-aa2c186f01dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:09.586935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68260996-c758-4434-b41e-2b2ce4838efe None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.620615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 300/603] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:09.621639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07df91c-f81d-4885-9f8d-aa2c186f01dc None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:09.636931 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434235f-d8d0-48bc-adfb-c2db697c30e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.638652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434235f-d8d0-48bc-adfb-c2db697c30e9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.639174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434235f-d8d0-48bc-adfb-c2db697c30e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.640054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434235f-d8d0-48bc-adfb-c2db697c30e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.641350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434235f-d8d0-48bc-adfb-c2db697c30e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:09.641844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434235f-d8d0-48bc-adfb-c2db697c30e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:09.643522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434235f-d8d0-48bc-adfb-c2db697c30e9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:09.662757 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 304/604] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:09.664523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434235f-d8d0-48bc-adfb-c2db697c30e9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:09.671596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:09.747969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07df91c-f81d-4885-9f8d-aa2c186f01dc None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.759094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:09.790005 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95d3fe-b4da-4b83-8b45-1952619bfe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.790626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95d3fe-b4da-4b83-8b45-1952619bfe8b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.790899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95d3fe-b4da-4b83-8b45-1952619bfe8b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.791197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95d3fe-b4da-4b83-8b45-1952619bfe8b None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.791937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95d3fe-b4da-4b83-8b45-1952619bfe8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:09.792233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95d3fe-b4da-4b83-8b45-1952619bfe8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:09.793754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95d3fe-b4da-4b83-8b45-1952619bfe8b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:09.801720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434235f-d8d0-48bc-adfb-c2db697c30e9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.807180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 301/605] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:09.807997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95d3fe-b4da-4b83-8b45-1952619bfe8b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:09.838537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e5606-8d05-4189-b308-7c769ad19319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.839416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e5606-8d05-4189-b308-7c769ad19319 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.839805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e5606-8d05-4189-b308-7c769ad19319 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.840182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e5606-8d05-4189-b308-7c769ad19319 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.851072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e5606-8d05-4189-b308-7c769ad19319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-5c6e5606-8d05-4189-b308-7c769ad19319 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:09.857027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:09.858126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:09.859222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:09.861725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 305/606] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:09.865454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e5606-8d05-4189-b308-7c769ad19319 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:09.916522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:09.920456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95d3fe-b4da-4b83-8b45-1952619bfe8b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.957172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eb9d7-84a2-4833-9ae0-89a80bbd399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:09.957852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eb9d7-84a2-4833-9ae0-89a80bbd399f None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:09.958126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eb9d7-84a2-4833-9ae0-89a80bbd399f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:09.958394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eb9d7-84a2-4833-9ae0-89a80bbd399f None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/f1e3ba8d0c7d4f8190cd0b5b057f5638/roles/bacea3b4b26e48a598b75ea1b301fafb` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:09.967127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e5606-8d05-4189-b308-7c769ad19319 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:09.973899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223eb9d7-84a2-4833-9ae0-89a80bbd399f None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=f1e3ba8d0c7d4f8190cd0b5b057f5638, role_id=bacea3b4b26e48a598b75ea1b301fafb)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:09.975452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223eb9d7-84a2-4833-9ae0-89a80bbd399f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:10.000745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 302/607] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:51:09 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/f1e3ba8d0c7d4f8190cd0b5b057f5638/roles/bacea3b4b26e48a598b75ea1b301fafb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:51:10.001670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eb9d7-84a2-4833-9ae0-89a80bbd399f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:10.005793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:10.013533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5744d-4260-4c9b-b0bb-4df6cb21f2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:10.014346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5744d-4260-4c9b-b0bb-4df6cb21f2af None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:10.014717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5744d-4260-4c9b-b0bb-4df6cb21f2af None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:10.015132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5744d-4260-4c9b-b0bb-4df6cb21f2af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:10.034012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5744d-4260-4c9b-b0bb-4df6cb21f2af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:10.035810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5744d-4260-4c9b-b0bb-4df6cb21f2af None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-39f5744d-4260-4c9b-b0bb-4df6cb21f2af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:10.061162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:10.062308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:10.063606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:10.063606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:10.063606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:10.063606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:10.063606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:10.063606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:10.063606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:10.063606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:10.063606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 306/608] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:10.068292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5744d-4260-4c9b-b0bb-4df6cb21f2af None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:10.178729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eb9d7-84a2-4833-9ae0-89a80bbd399f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:10.233868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d25330-9dc2-44fd-9795-af06c9a657e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:10.234718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d25330-9dc2-44fd-9795-af06c9a657e1 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:10.235137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d25330-9dc2-44fd-9795-af06c9a657e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:10.235852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d25330-9dc2-44fd-9795-af06c9a657e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:10.236526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d25330-9dc2-44fd-9795-af06c9a657e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:10.237109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d25330-9dc2-44fd-9795-af06c9a657e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:10.238661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d25330-9dc2-44fd-9795-af06c9a657e1 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:10.248865 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5744d-4260-4c9b-b0bb-4df6cb21f2af None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:10.250008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 303/609] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:10.256719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d25330-9dc2-44fd-9795-af06c9a657e1 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:10.273710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:10.292620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a579279-0fa1-4d33-8c95-b9007118ff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:10.293336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a579279-0fa1-4d33-8c95-b9007118ff50 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:10.293691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a579279-0fa1-4d33-8c95-b9007118ff50 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:10.294039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a579279-0fa1-4d33-8c95-b9007118ff50 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:10.294811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a579279-0fa1-4d33-8c95-b9007118ff50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:10.295223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a579279-0fa1-4d33-8c95-b9007118ff50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:10.296714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a579279-0fa1-4d33-8c95-b9007118ff50 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:10.311041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 307/610] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:10.317084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a579279-0fa1-4d33-8c95-b9007118ff50 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:10.417682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d25330-9dc2-44fd-9795-af06c9a657e1 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:10.423438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:10.483872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598e685-f250-4f11-a8b9-b77335a2b5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:10.484513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598e685-f250-4f11-a8b9-b77335a2b5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:10.484791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598e685-f250-4f11-a8b9-b77335a2b5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:10.485065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598e685-f250-4f11-a8b9-b77335a2b5c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:10.504181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a579279-0fa1-4d33-8c95-b9007118ff50 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:10.508563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598e685-f250-4f11-a8b9-b77335a2b5c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:10.509949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598e685-f250-4f11-a8b9-b77335a2b5c5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:10.539810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd9df7-e4e6-4a90-b038-029b9f811a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:10.540474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd9df7-e4e6-4a90-b038-029b9f811a50 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:10.540756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd9df7-e4e6-4a90-b038-029b9f811a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:10.541034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd9df7-e4e6-4a90-b038-029b9f811a50 None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/ab9b7576a3834862af6abb7e0416f158/roles/bacea3b4b26e48a598b75ea1b301fafb` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-c598e685-f250-4f11-a8b9-b77335a2b5c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:10.544026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:10.544954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:10.546412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:10.546412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:10.546412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:10.546412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 304/611] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:10.550622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598e685-f250-4f11-a8b9-b77335a2b5c5 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:10.556928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd9df7-e4e6-4a90-b038-029b9f811a50 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=ab9b7576a3834862af6abb7e0416f158, role_id=bacea3b4b26e48a598b75ea1b301fafb)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:10.558607 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd9df7-e4e6-4a90-b038-029b9f811a50 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:10.574580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 308/612] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:51:10 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/ab9b7576a3834862af6abb7e0416f158/roles/bacea3b4b26e48a598b75ea1b301fafb => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:51:10.681741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:10.731226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598e685-f250-4f11-a8b9-b77335a2b5c5 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:10.784672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17e7d7-245c-417e-8e89-7a67dfd4e6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:10.785282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17e7d7-245c-417e-8e89-7a67dfd4e6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:10.785497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17e7d7-245c-417e-8e89-7a67dfd4e6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:10.785718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17e7d7-245c-417e-8e89-7a67dfd4e6e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:10.786373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17e7d7-245c-417e-8e89-7a67dfd4e6e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:10.786668 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17e7d7-245c-417e-8e89-7a67dfd4e6e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:10.788085 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17e7d7-245c-417e-8e89-7a67dfd4e6e4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:10.813005 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 305/613] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:10.818679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd9df7-e4e6-4a90-b038-029b9f811a50 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:10.870374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d04a2-bc39-4b82-a5aa-e1550a5bebf9 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:10.870721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d04a2-bc39-4b82-a5aa-e1550a5bebf9 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:10.870910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d04a2-bc39-4b82-a5aa-e1550a5bebf9 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:10.872570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 306/614] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:10.877840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc622236-1081-4910-b405-d62bcc1a2736 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:10.878119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc622236-1081-4910-b405-d62bcc1a2736 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:10.878388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc622236-1081-4910-b405-d62bcc1a2736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:10.898881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-bc622236-1081-4910-b405-d62bcc1a2736 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:10.901013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bc622236-1081-4910-b405-d62bcc1a2736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:10.951315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd9df7-e4e6-4a90-b038-029b9f811a50 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:10.963408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:10.996571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 307/615] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:51:11.010499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b29524-ca6b-428b-aa68-9b226e37c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:11.011200 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b29524-ca6b-428b-aa68-9b226e37c808 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.011483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b29524-ca6b-428b-aa68-9b226e37c808 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.011844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b29524-ca6b-428b-aa68-9b226e37c808 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.022254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b29524-ca6b-428b-aa68-9b226e37c808 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:11.024396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b29524-ca6b-428b-aa68-9b226e37c808 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-19b29524-ca6b-428b-aa68-9b226e37c808 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:11.038942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:11.039859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:11.041352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 309/616] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:11.045717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc622236-1081-4910-b405-d62bcc1a2736 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:11.060034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b29524-ca6b-428b-aa68-9b226e37c808 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:11.199153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b29524-ca6b-428b-aa68-9b226e37c808 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:11.200584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:11.252704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ace1b-29ab-4ecb-a572-69c299d32a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:11.253342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ace1b-29ab-4ecb-a572-69c299d32a5a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.253624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ace1b-29ab-4ecb-a572-69c299d32a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.253901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ace1b-29ab-4ecb-a572-69c299d32a5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.261006 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc622236-1081-4910-b405-d62bcc1a2736 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:11.274271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ace1b-29ab-4ecb-a572-69c299d32a5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-aa9ace1b-29ab-4ecb-a572-69c299d32a5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:11.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:11.287685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:11.289153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 310/617] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:11.295977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ace1b-29ab-4ecb-a572-69c299d32a5a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:11.300859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca5d8b-93a2-4d96-9aa1-5c5482682319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:11.301476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca5d8b-93a2-4d96-9aa1-5c5482682319 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.301749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca5d8b-93a2-4d96-9aa1-5c5482682319 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.302014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca5d8b-93a2-4d96-9aa1-5c5482682319 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.302741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca5d8b-93a2-4d96-9aa1-5c5482682319 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:11.303067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca5d8b-93a2-4d96-9aa1-5c5482682319 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:11.305068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca5d8b-93a2-4d96-9aa1-5c5482682319 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:11.328584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 308/618] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 10 09:51:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:11.333761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca5d8b-93a2-4d96-9aa1-5c5482682319 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:11.339344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:11.444768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca5d8b-93a2-4d96-9aa1-5c5482682319 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:11.463306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:11.483595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83b866-9961-4475-861a-5b1eec64ca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:11.484786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b866-9961-4475-861a-5b1eec64ca5d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.485093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b866-9961-4475-861a-5b1eec64ca5d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.485376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b866-9961-4475-861a-5b1eec64ca5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.497146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83b866-9961-4475-861a-5b1eec64ca5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:11.498594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83b866-9961-4475-861a-5b1eec64ca5d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-7c83b866-9961-4475-861a-5b1eec64ca5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:11.515154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:11.516099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:11.517714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:11.517714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:11.517714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:11.517714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 309/619] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:11.527943 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83b866-9961-4475-861a-5b1eec64ca5d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:11.547650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ace1b-29ab-4ecb-a572-69c299d32a5a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:11.587208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c3a13-a36d-4e43-b44a-afa8845b8e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:11.587878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c3a13-a36d-4e43-b44a-afa8845b8e85 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.588160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c3a13-a36d-4e43-b44a-afa8845b8e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.588535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c3a13-a36d-4e43-b44a-afa8845b8e85 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.589359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c3a13-a36d-4e43-b44a-afa8845b8e85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:11.589668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c3a13-a36d-4e43-b44a-afa8845b8e85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:11.591063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c3a13-a36d-4e43-b44a-afa8845b8e85 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:11.613196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 311/620] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:11.623909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c3a13-a36d-4e43-b44a-afa8845b8e85 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:11.659930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83b866-9961-4475-861a-5b1eec64ca5d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:11.693944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11339ca-6c89-43f1-9e95-208bd2df0673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:11.694547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11339ca-6c89-43f1-9e95-208bd2df0673 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.694825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11339ca-6c89-43f1-9e95-208bd2df0673 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.695635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11339ca-6c89-43f1-9e95-208bd2df0673 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.696270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11339ca-6c89-43f1-9e95-208bd2df0673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:11.696568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11339ca-6c89-43f1-9e95-208bd2df0673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:11.697919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11339ca-6c89-43f1-9e95-208bd2df0673 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:11.709966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 310/621] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:11.719889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11339ca-6c89-43f1-9e95-208bd2df0673 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:11.747348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:11.817849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11339ca-6c89-43f1-9e95-208bd2df0673 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:11.829380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:11.851881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c3a13-a36d-4e43-b44a-afa8845b8e85 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:11.872878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5cfd2-ac7c-4c54-a523-28bb25b06693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:11.873516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5cfd2-ac7c-4c54-a523-28bb25b06693 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.873791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5cfd2-ac7c-4c54-a523-28bb25b06693 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.874061 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5cfd2-ac7c-4c54-a523-28bb25b06693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.891052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5cfd2-ac7c-4c54-a523-28bb25b06693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:11.892511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5cfd2-ac7c-4c54-a523-28bb25b06693 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-1da5cfd2-ac7c-4c54-a523-28bb25b06693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:11.909276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:11.910237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:11.911160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:11.911160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:11.911160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:11.911160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:11.911160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:11.911160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:11.911160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:11.911160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:11.911160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 311/622] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:11.911655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2523ba5-c5ca-437a-af6c-44e99833c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:11.912251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76068c2-e289-4733-b13e-946b06be596e None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.912351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2523ba5-c5ca-437a-af6c-44e99833c201 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.912435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76068c2-e289-4733-b13e-946b06be596e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.912512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2523ba5-c5ca-437a-af6c-44e99833c201 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.912580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76068c2-e289-4733-b13e-946b06be596e None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.912646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2523ba5-c5ca-437a-af6c-44e99833c201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.913989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 312/623] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:11.918998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76068c2-e289-4733-b13e-946b06be596e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:11.928676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2523ba5-c5ca-437a-af6c-44e99833c201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:11.930513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2523ba5-c5ca-437a-af6c-44e99833c201 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-a2523ba5-c5ca-437a-af6c-44e99833c201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:11.949562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:11.950711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:11.951738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:11.951738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:11.951738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:11.955017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 312/624] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:11.956408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb5bfe-b088-4547-a0a6-250f7c625374 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:11.956690 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb5bfe-b088-4547-a0a6-250f7c625374 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:11.956963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb5bfe-b088-4547-a0a6-250f7c625374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:11.975538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2dcb5bfe-b088-4547-a0a6-250f7c625374 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:11.976756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2dcb5bfe-b088-4547-a0a6-250f7c625374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:12.037719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:12.056436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76068c2-e289-4733-b13e-946b06be596e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:12.057446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 313/625] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:12.058499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb5bfe-b088-4547-a0a6-250f7c625374 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:12.108311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b1147-c093-42ca-b7f4-26beb64342c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:12.109212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b1147-c093-42ca-b7f4-26beb64342c1 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.109624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b1147-c093-42ca-b7f4-26beb64342c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.110022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b1147-c093-42ca-b7f4-26beb64342c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.110867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b1147-c093-42ca-b7f4-26beb64342c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:12.111352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b1147-c093-42ca-b7f4-26beb64342c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:12.112943 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b1147-c093-42ca-b7f4-26beb64342c1 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:12.126563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 313/626] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:12.132538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b1147-c093-42ca-b7f4-26beb64342c1 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:12.144090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:12.180964 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb5bfe-b088-4547-a0a6-250f7c625374 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:12.216702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bcc3b-703b-4129-a069-531e823b9d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:12.217473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bcc3b-703b-4129-a069-531e823b9d35 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.217849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bcc3b-703b-4129-a069-531e823b9d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.218212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bcc3b-703b-4129-a069-531e823b9d35 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.219085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bcc3b-703b-4129-a069-531e823b9d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:12.219484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bcc3b-703b-4129-a069-531e823b9d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:12.221020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bcc3b-703b-4129-a069-531e823b9d35 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:12.238206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 314/627] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:12.239177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bcc3b-703b-4129-a069-531e823b9d35 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:12.239577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b1147-c093-42ca-b7f4-26beb64342c1 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:12.277052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8583d-a399-4330-86ef-bd71ff5798bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:12.278152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8583d-a399-4330-86ef-bd71ff5798bc None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.278606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8583d-a399-4330-86ef-bd71ff5798bc None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.279131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8583d-a399-4330-86ef-bd71ff5798bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.290565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8583d-a399-4330-86ef-bd71ff5798bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-9ca8583d-a399-4330-86ef-bd71ff5798bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:12.296502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:12.297399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:12.298222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:12.300363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 314/628] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:12.306030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8583d-a399-4330-86ef-bd71ff5798bc None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:12.310487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:12.346935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bcc3b-703b-4129-a069-531e823b9d35 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:12.373262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:12.384849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a781c0-d38f-4a13-9cd5-8d1b082dd132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:12.385510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a781c0-d38f-4a13-9cd5-8d1b082dd132 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.385791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a781c0-d38f-4a13-9cd5-8d1b082dd132 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.386067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a781c0-d38f-4a13-9cd5-8d1b082dd132 None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/af29f3451f654f53ac98239a2b28d128/roles/bacea3b4b26e48a598b75ea1b301fafb` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.401456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a781c0-d38f-4a13-9cd5-8d1b082dd132 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=af29f3451f654f53ac98239a2b28d128, role_id=bacea3b4b26e48a598b75ea1b301fafb)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:12.402911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a781c0-d38f-4a13-9cd5-8d1b082dd132 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:12.419603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 315/629] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:51:12 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/af29f3451f654f53ac98239a2b28d128/roles/bacea3b4b26e48a598b75ea1b301fafb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:51:12.420347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a781c0-d38f-4a13-9cd5-8d1b082dd132 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:12.431694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8583d-a399-4330-86ef-bd71ff5798bc None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:12.481722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704829c9-5ac0-4c79-b3d0-5eee4f1519b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:12.482375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704829c9-5ac0-4c79-b3d0-5eee4f1519b7 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.482663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704829c9-5ac0-4c79-b3d0-5eee4f1519b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.482939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704829c9-5ac0-4c79-b3d0-5eee4f1519b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.492649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704829c9-5ac0-4c79-b3d0-5eee4f1519b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:12.494553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704829c9-5ac0-4c79-b3d0-5eee4f1519b7 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-704829c9-5ac0-4c79-b3d0-5eee4f1519b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:12.509124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:12.510144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 09:51:12.511096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:12.515311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 315/630] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:12.516777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1a5ae-1d5b-4c1a-b299-9d8dbd4a7c8d None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.517116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1a5ae-1d5b-4c1a-b299-9d8dbd4a7c8d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.518189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1a5ae-1d5b-4c1a-b299-9d8dbd4a7c8d None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.523669 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 316/631] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:12.524237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1a5ae-1d5b-4c1a-b299-9d8dbd4a7c8d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:12.559771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a781c0-d38f-4a13-9cd5-8d1b082dd132 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:12.600065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:12.605121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289c65f-63b7-4010-bb49-8d2e84d9351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:12.605918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289c65f-63b7-4010-bb49-8d2e84d9351b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.606337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289c65f-63b7-4010-bb49-8d2e84d9351b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.606744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289c65f-63b7-4010-bb49-8d2e84d9351b None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.607632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289c65f-63b7-4010-bb49-8d2e84d9351b None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:12.608089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289c65f-63b7-4010-bb49-8d2e84d9351b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:12.609589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289c65f-63b7-4010-bb49-8d2e84d9351b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:12.619454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 316/632] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:12.620756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49655b-3514-44e6-b8bb-b5737f0f401e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.621040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49655b-3514-44e6-b8bb-b5737f0f401e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.621303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49655b-3514-44e6-b8bb-b5737f0f401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.647210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3c49655b-3514-44e6-b8bb-b5737f0f401e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:12.648454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1a5ae-1d5b-4c1a-b299-9d8dbd4a7c8d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:12.649004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3c49655b-3514-44e6-b8bb-b5737f0f401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:12.686714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05654d-4e24-4d25-882a-19a1f340247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:12.687358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05654d-4e24-4d25-882a-19a1f340247b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.687636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05654d-4e24-4d25-882a-19a1f340247b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.687908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05654d-4e24-4d25-882a-19a1f340247b None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.688634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05654d-4e24-4d25-882a-19a1f340247b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:12.689072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05654d-4e24-4d25-882a-19a1f340247b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:12.690221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:12.691698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05654d-4e24-4d25-882a-19a1f340247b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:12.717422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 317/633] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 10 09:51:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:12.717422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05654d-4e24-4d25-882a-19a1f340247b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:12.744001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 317/634] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:12.745111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49655b-3514-44e6-b8bb-b5737f0f401e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:12.846666 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05654d-4e24-4d25-882a-19a1f340247b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:12.872787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:12.886275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b32d0f-c86f-4bbd-b7f1-09c1b380a031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:12.887109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b32d0f-c86f-4bbd-b7f1-09c1b380a031 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.887489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b32d0f-c86f-4bbd-b7f1-09c1b380a031 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.887896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b32d0f-c86f-4bbd-b7f1-09c1b380a031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.895802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49655b-3514-44e6-b8bb-b5737f0f401e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:12.899799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b32d0f-c86f-4bbd-b7f1-09c1b380a031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:12.901475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b32d0f-c86f-4bbd-b7f1-09c1b380a031 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-54b32d0f-c86f-4bbd-b7f1-09c1b380a031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:12.920108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:12.921268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:12.922333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:12.922333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:12.922333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:12.924523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 318/635] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:12.925607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b32d0f-c86f-4bbd-b7f1-09c1b380a031 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:12.946117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc6374-0032-414b-a687-a37d81e8beef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:12.946984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc6374-0032-414b-a687-a37d81e8beef None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:12.947668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc6374-0032-414b-a687-a37d81e8beef None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:12.948090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc6374-0032-414b-a687-a37d81e8beef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:12.995196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:13.031658 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc6374-0032-414b-a687-a37d81e8beef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:13.033467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc6374-0032-414b-a687-a37d81e8beef None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:13.040151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b32d0f-c86f-4bbd-b7f1-09c1b380a031 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.076442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e367886-2ab3-4f90-b687-31de6ad3136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.077064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e367886-2ab3-4f90-b687-31de6ad3136e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.077566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e367886-2ab3-4f90-b687-31de6ad3136e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.077848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e367886-2ab3-4f90-b687-31de6ad3136e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.089057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e367886-2ab3-4f90-b687-31de6ad3136e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-1e367886-2ab3-4f90-b687-31de6ad3136e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:13.094519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:13.095477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:13.096314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 319/636] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:13.097073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e367886-2ab3-4f90-b687-31de6ad3136e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.137819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 318/637] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:13.139234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da574c6-1c2b-4e80-bf80-0d3469df47ca None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.139566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da574c6-1c2b-4e80-bf80-0d3469df47ca None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.139910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da574c6-1c2b-4e80-bf80-0d3469df47ca None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.143064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 319/638] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:13.143064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da574c6-1c2b-4e80-bf80-0d3469df47ca None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.143287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:13.216322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:13.217294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e367886-2ab3-4f90-b687-31de6ad3136e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.251726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4272e-875c-47c5-be6b-24ebfbea60ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.252322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4272e-875c-47c5-be6b-24ebfbea60ea None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.252600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4272e-875c-47c5-be6b-24ebfbea60ea None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.252964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4272e-875c-47c5-be6b-24ebfbea60ea None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.253703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4272e-875c-47c5-be6b-24ebfbea60ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:13.253986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4272e-875c-47c5-be6b-24ebfbea60ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:13.254385 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da574c6-1c2b-4e80-bf80-0d3469df47ca None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.255376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4272e-875c-47c5-be6b-24ebfbea60ea None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:13.272389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 320/639] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:13.273872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4272e-875c-47c5-be6b-24ebfbea60ea None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.291727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c505c-d6e2-47b9-985d-69d6c7ee5b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.292338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c505c-d6e2-47b9-985d-69d6c7ee5b9c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.292620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c505c-d6e2-47b9-985d-69d6c7ee5b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.292892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c505c-d6e2-47b9-985d-69d6c7ee5b9c None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.293552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c505c-d6e2-47b9-985d-69d6c7ee5b9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:13.293857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c505c-d6e2-47b9-985d-69d6c7ee5b9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:13.295253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c505c-d6e2-47b9-985d-69d6c7ee5b9c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:13.303272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 320/640] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:13.304333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf2a3-57b4-4fba-8e6b-e511c3f88ab0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.304619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf2a3-57b4-4fba-8e6b-e511c3f88ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.304905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf2a3-57b4-4fba-8e6b-e511c3f88ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.322660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cefaf2a3-57b4-4fba-8e6b-e511c3f88ab0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:13.323875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cefaf2a3-57b4-4fba-8e6b-e511c3f88ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:13.377908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:13.385846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4272e-875c-47c5-be6b-24ebfbea60ea None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.400942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 321/641] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:13.400942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf2a3-57b4-4fba-8e6b-e511c3f88ab0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.434940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd865b-77a7-42db-b350-ff9e2d2f957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.435693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd865b-77a7-42db-b350-ff9e2d2f957c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.435967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd865b-77a7-42db-b350-ff9e2d2f957c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.436252 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd865b-77a7-42db-b350-ff9e2d2f957c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.447495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd865b-77a7-42db-b350-ff9e2d2f957c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:13.448922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd865b-77a7-42db-b350-ff9e2d2f957c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-01dd865b-77a7-42db-b350-ff9e2d2f957c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:13.464047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:13.464971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:13.465862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:13.465862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:13.465862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:13.465862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 321/642] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:13.465862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd865b-77a7-42db-b350-ff9e2d2f957c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.472050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:13.514894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf2a3-57b4-4fba-8e6b-e511c3f88ab0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.550790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022fe3f-6f15-48b2-915a-64a2e84ac6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.551471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022fe3f-6f15-48b2-915a-64a2e84ac6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.551750 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022fe3f-6f15-48b2-915a-64a2e84ac6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.552020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022fe3f-6f15-48b2-915a-64a2e84ac6a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.560540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022fe3f-6f15-48b2-915a-64a2e84ac6a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:13.562276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022fe3f-6f15-48b2-915a-64a2e84ac6a9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-3022fe3f-6f15-48b2-915a-64a2e84ac6a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:13.575771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:13.581661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 322/643] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:13.582604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022fe3f-6f15-48b2-915a-64a2e84ac6a9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.588326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd865b-77a7-42db-b350-ff9e2d2f957c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.625458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab607ba-dbf0-45bb-894b-9d5f672770e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.626082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab607ba-dbf0-45bb-894b-9d5f672770e8 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.626361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab607ba-dbf0-45bb-894b-9d5f672770e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.626632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab607ba-dbf0-45bb-894b-9d5f672770e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.638702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab607ba-dbf0-45bb-894b-9d5f672770e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:13.640353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab607ba-dbf0-45bb-894b-9d5f672770e8 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:13.647116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-aab607ba-dbf0-45bb-894b-9d5f672770e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:13.656924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:13.657883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:13.658972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:13.658972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:13.658972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:13.658972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 322/644] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:13.658972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab607ba-dbf0-45bb-894b-9d5f672770e8 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.686743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022fe3f-6f15-48b2-915a-64a2e84ac6a9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.709288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:13.723256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22fd5f-4da2-4ff9-af3a-5fe994059183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.723813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22fd5f-4da2-4ff9-af3a-5fe994059183 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.724091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22fd5f-4da2-4ff9-af3a-5fe994059183 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.724366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22fd5f-4da2-4ff9-af3a-5fe994059183 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.734656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22fd5f-4da2-4ff9-af3a-5fe994059183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ba22fd5f-4da2-4ff9-af3a-5fe994059183 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:13.739652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:13.740762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:13.742343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:13.746136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 323/645] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:13.747377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22fd5f-4da2-4ff9-af3a-5fe994059183 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.771436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab607ba-dbf0-45bb-894b-9d5f672770e8 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.807711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613562e1-16dd-4f18-a041-4f165bacf663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.808308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613562e1-16dd-4f18-a041-4f165bacf663 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.808587 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613562e1-16dd-4f18-a041-4f165bacf663 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.808858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613562e1-16dd-4f18-a041-4f165bacf663 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.809490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613562e1-16dd-4f18-a041-4f165bacf663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:13.809785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613562e1-16dd-4f18-a041-4f165bacf663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:13.811176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613562e1-16dd-4f18-a041-4f165bacf663 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:13.823798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 323/646] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:13.828716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613562e1-16dd-4f18-a041-4f165bacf663 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.846683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22fd5f-4da2-4ff9-af3a-5fe994059183 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.881224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7935d3e-3c5e-4fbd-a2cc-362f2153ae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.881834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7935d3e-3c5e-4fbd-a2cc-362f2153ae38 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.882106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7935d3e-3c5e-4fbd-a2cc-362f2153ae38 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.882431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7935d3e-3c5e-4fbd-a2cc-362f2153ae38 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.883676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7935d3e-3c5e-4fbd-a2cc-362f2153ae38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:13.883982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7935d3e-3c5e-4fbd-a2cc-362f2153ae38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:13.885392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7935d3e-3c5e-4fbd-a2cc-362f2153ae38 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:13.888970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:13.902767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 324/647] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:13.903680 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7935d3e-3c5e-4fbd-a2cc-362f2153ae38 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:13.935734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613562e1-16dd-4f18-a041-4f165bacf663 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:13.942777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:13.971142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c69c63-b3ef-48e1-b54c-e7a29199fc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:13.971746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c69c63-b3ef-48e1-b54c-e7a29199fc0b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:13.972048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c69c63-b3ef-48e1-b54c-e7a29199fc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:13.972300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c69c63-b3ef-48e1-b54c-e7a29199fc0b None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:13.972931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c69c63-b3ef-48e1-b54c-e7a29199fc0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:13.973228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c69c63-b3ef-48e1-b54c-e7a29199fc0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:13.974643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c69c63-b3ef-48e1-b54c-e7a29199fc0b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:13.990688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 324/648] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:13.990688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c69c63-b3ef-48e1-b54c-e7a29199fc0b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.010691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7935d3e-3c5e-4fbd-a2cc-362f2153ae38 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.052130 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1beff3-db31-4af3-958f-5d526f63d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.052852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1beff3-db31-4af3-958f-5d526f63d338 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.053195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1beff3-db31-4af3-958f-5d526f63d338 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.053536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1beff3-db31-4af3-958f-5d526f63d338 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.054313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1beff3-db31-4af3-958f-5d526f63d338 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:14.055193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1beff3-db31-4af3-958f-5d526f63d338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.056604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1beff3-db31-4af3-958f-5d526f63d338 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.064737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 325/649] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:14.069964 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1beff3-db31-4af3-958f-5d526f63d338 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.098245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c69c63-b3ef-48e1-b54c-e7a29199fc0b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.116344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:14.137450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc93cbe-1a33-40e2-b90b-d966ef4c0a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.138174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc93cbe-1a33-40e2-b90b-d966ef4c0a20 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.138526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc93cbe-1a33-40e2-b90b-d966ef4c0a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.138866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc93cbe-1a33-40e2-b90b-d966ef4c0a20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.145750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc93cbe-1a33-40e2-b90b-d966ef4c0a20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.147318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc93cbe-1a33-40e2-b90b-d966ef4c0a20 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-dbc93cbe-1a33-40e2-b90b-d966ef4c0a20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:14.158357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:14.159500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:14.160628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:14.160628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:14.160628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:14.160628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:14.160628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:14.160628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:14.160628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:14.160628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:14.161165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 325/650] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:14.166928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc93cbe-1a33-40e2-b90b-d966ef4c0a20 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.170067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:14.173214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1beff3-db31-4af3-958f-5d526f63d338 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.213199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b49cd-e5c2-4add-a05b-a90e03e7895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.213807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b49cd-e5c2-4add-a05b-a90e03e7895b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.214085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b49cd-e5c2-4add-a05b-a90e03e7895b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.214387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b49cd-e5c2-4add-a05b-a90e03e7895b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.270759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc93cbe-1a33-40e2-b90b-d966ef4c0a20 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.284492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b49cd-e5c2-4add-a05b-a90e03e7895b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.285911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b49cd-e5c2-4add-a05b-a90e03e7895b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.307668 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad17d38-be0e-441d-b71e-f823c80c3c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.308295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad17d38-be0e-441d-b71e-f823c80c3c0b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.308565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad17d38-be0e-441d-b71e-f823c80c3c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.308834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad17d38-be0e-441d-b71e-f823c80c3c0b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.316853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad17d38-be0e-441d-b71e-f823c80c3c0b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.318290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad17d38-be0e-441d-b71e-f823c80c3c0b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-9ad17d38-be0e-441d-b71e-f823c80c3c0b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:14.329644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:14.330777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 09:51:14.332032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:14.335311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 326/651] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:14.336651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad17d38-be0e-441d-b71e-f823c80c3c0b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.378116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:14.393661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:14.403337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 326/652] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:14.404373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b49cd-e5c2-4add-a05b-a90e03e7895b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.452979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad17d38-be0e-441d-b71e-f823c80c3c0b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.486082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa726ec1-0352-46ea-b583-97ddacfbc1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.486707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa726ec1-0352-46ea-b583-97ddacfbc1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.487013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa726ec1-0352-46ea-b583-97ddacfbc1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.487289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa726ec1-0352-46ea-b583-97ddacfbc1a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.498173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa726ec1-0352-46ea-b583-97ddacfbc1a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.499675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa726ec1-0352-46ea-b583-97ddacfbc1a1 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.505550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b49cd-e5c2-4add-a05b-a90e03e7895b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-aa726ec1-0352-46ea-b583-97ddacfbc1a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:14.513324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:14.514414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:14.515342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:14.515342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:14.515342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:14.515342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 327/653] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:14.515342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa726ec1-0352-46ea-b583-97ddacfbc1a1 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.540771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de4ef2-e288-46dd-801e-7af452052c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.541370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de4ef2-e288-46dd-801e-7af452052c48 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.541642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de4ef2-e288-46dd-801e-7af452052c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.541912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de4ef2-e288-46dd-801e-7af452052c48 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.542600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de4ef2-e288-46dd-801e-7af452052c48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:14.542926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de4ef2-e288-46dd-801e-7af452052c48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.544347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de4ef2-e288-46dd-801e-7af452052c48 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.555777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 327/654] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:14.556442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de4ef2-e288-46dd-801e-7af452052c48 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.603232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:14.613338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa726ec1-0352-46ea-b583-97ddacfbc1a1 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.613861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:14.645423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fb27c-676d-4665-bbc0-cb2e2a83a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.646012 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fb27c-676d-4665-bbc0-cb2e2a83a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.646280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fb27c-676d-4665-bbc0-cb2e2a83a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.646542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fb27c-676d-4665-bbc0-cb2e2a83a4e3 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.647298 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fb27c-676d-4665-bbc0-cb2e2a83a4e3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:14.647594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fb27c-676d-4665-bbc0-cb2e2a83a4e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.649517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fb27c-676d-4665-bbc0-cb2e2a83a4e3 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.653921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de4ef2-e288-46dd-801e-7af452052c48 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.666484 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 328/655] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:14.667650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a1730-0e4e-49ae-9c6b-84f0815a655a None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.667931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a1730-0e4e-49ae-9c6b-84f0815a655a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.668200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a1730-0e4e-49ae-9c6b-84f0815a655a None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.669372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 329/656] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:14.670057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a1730-0e4e-49ae-9c6b-84f0815a655a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.691530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77d53f-a145-46a3-94d5-ae8ebc1284b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.692215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d53f-a145-46a3-94d5-ae8ebc1284b2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.692528 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d53f-a145-46a3-94d5-ae8ebc1284b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.692908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d53f-a145-46a3-94d5-ae8ebc1284b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.703859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77d53f-a145-46a3-94d5-ae8ebc1284b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.705308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77d53f-a145-46a3-94d5-ae8ebc1284b2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0d77d53f-a145-46a3-94d5-ae8ebc1284b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:14.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:14.720878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:14.721749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:14.721749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:14.721749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:14.721749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 328/657] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:14.721971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77d53f-a145-46a3-94d5-ae8ebc1284b2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.776528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a1730-0e4e-49ae-9c6b-84f0815a655a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.809503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1209017-4585-455a-ac22-024074121b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.810024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1209017-4585-455a-ac22-024074121b5b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.810299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1209017-4585-455a-ac22-024074121b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.810565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1209017-4585-455a-ac22-024074121b5b None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.811227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1209017-4585-455a-ac22-024074121b5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:14.811521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1209017-4585-455a-ac22-024074121b5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.812886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1209017-4585-455a-ac22-024074121b5b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.820491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77d53f-a145-46a3-94d5-ae8ebc1284b2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.824113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 330/658] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:14.824890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1209017-4585-455a-ac22-024074121b5b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.833526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:14.840263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:14.854874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fc0a9-8c74-4d02-ba78-f2325c48750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:14.855501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fc0a9-8c74-4d02-ba78-f2325c48750a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.855774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fc0a9-8c74-4d02-ba78-f2325c48750a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.856039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fc0a9-8c74-4d02-ba78-f2325c48750a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.856746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fc0a9-8c74-4d02-ba78-f2325c48750a None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=b8f456f0d15b4d068cfcd6476563247f) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:14.857047 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fc0a9-8c74-4d02-ba78-f2325c48750a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:14.858460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fc0a9-8c74-4d02-ba78-f2325c48750a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:14.873506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 329/659] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/role_assignments?user.id=b8f456f0d15b4d068cfcd6476563247f&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:14.873506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707ac4-18d6-4e1b-b03d-4e0fe8a4d196 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:14.873506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707ac4-18d6-4e1b-b03d-4e0fe8a4d196 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:14.873506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707ac4-18d6-4e1b-b03d-4e0fe8a4d196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:14.885315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-15707ac4-18d6-4e1b-b03d-4e0fe8a4d196 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:14.886646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-15707ac4-18d6-4e1b-b03d-4e0fe8a4d196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:14.958199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1209017-4585-455a-ac22-024074121b5b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:14.991013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 330/660] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:14.991013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707ac4-18d6-4e1b-b03d-4e0fe8a4d196 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:14.999611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e38a0f-b069-4220-a6ce-e7e20bc12484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.000516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e38a0f-b069-4220-a6ce-e7e20bc12484 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.000954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e38a0f-b069-4220-a6ce-e7e20bc12484 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.001409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e38a0f-b069-4220-a6ce-e7e20bc12484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.079999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e38a0f-b069-4220-a6ce-e7e20bc12484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:15.081583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e38a0f-b069-4220-a6ce-e7e20bc12484 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:15.092091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:15.116291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707ac4-18d6-4e1b-b03d-4e0fe8a4d196 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.124800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:15.154573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963139-57c4-45de-a6e5-f5ddaddf836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.155219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963139-57c4-45de-a6e5-f5ddaddf836a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.155501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963139-57c4-45de-a6e5-f5ddaddf836a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.155777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963139-57c4-45de-a6e5-f5ddaddf836a None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.156427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52963139-57c4-45de-a6e5-f5ddaddf836a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:15.156728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52963139-57c4-45de-a6e5-f5ddaddf836a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:15.158082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52963139-57c4-45de-a6e5-f5ddaddf836a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:15.169693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 331/661] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:15.170578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963139-57c4-45de-a6e5-f5ddaddf836a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.196777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 331/662] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:15.197630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e38a0f-b069-4220-a6ce-e7e20bc12484 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.283648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963139-57c4-45de-a6e5-f5ddaddf836a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.300714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e38a0f-b069-4220-a6ce-e7e20bc12484 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.311109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:15.318423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355a47e-c737-4742-a7ff-64a6195cb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.319098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355a47e-c737-4742-a7ff-64a6195cb3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.319376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355a47e-c737-4742-a7ff-64a6195cb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.319643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355a47e-c737-4742-a7ff-64a6195cb3a4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.327883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355a47e-c737-4742-a7ff-64a6195cb3a4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:15.329299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355a47e-c737-4742-a7ff-64a6195cb3a4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:15.338098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebc2b-ca9b-4d68-912a-eb7df7afecbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.338647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebc2b-ca9b-4d68-912a-eb7df7afecbd None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.338921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebc2b-ca9b-4d68-912a-eb7df7afecbd None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.339224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebc2b-ca9b-4d68-912a-eb7df7afecbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6355a47e-c737-4742-a7ff-64a6195cb3a4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:15.340923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:15.341898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 332/663] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:15.342822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355a47e-c737-4742-a7ff-64a6195cb3a4 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.355866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bebc2b-ca9b-4d68-912a-eb7df7afecbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-80bebc2b-ca9b-4d68-912a-eb7df7afecbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:15.360481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:15.361411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:15.362235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 332/664] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:15.363575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebc2b-ca9b-4d68-912a-eb7df7afecbd None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.379387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:15.467815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebc2b-ca9b-4d68-912a-eb7df7afecbd None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.468413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355a47e-c737-4742-a7ff-64a6195cb3a4 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.504629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea62944-e0f5-44ca-896f-ff20c137b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.505345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea62944-e0f5-44ca-896f-ff20c137b557 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.505634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea62944-e0f5-44ca-896f-ff20c137b557 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.505907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea62944-e0f5-44ca-896f-ff20c137b557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.506315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c984177-5e43-4c42-bb60-5160f3d2098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.507100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c984177-5e43-4c42-bb60-5160f3d2098f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.507478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c984177-5e43-4c42-bb60-5160f3d2098f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.507811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c984177-5e43-4c42-bb60-5160f3d2098f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.514458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c984177-5e43-4c42-bb60-5160f3d2098f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:15.515988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c984177-5e43-4c42-bb60-5160f3d2098f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:15.517045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea62944-e0f5-44ca-896f-ff20c137b557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:15.518547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea62944-e0f5-44ca-896f-ff20c137b557 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9c984177-5e43-4c42-bb60-5160f3d2098f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:15.527198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:15.528072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:15.528949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:15.528949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:15.528949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:15.528949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:15.528949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:15.528949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:15.528949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:15.528949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:15.530926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 333/665] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:15.531858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c984177-5e43-4c42-bb60-5160f3d2098f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-3ea62944-e0f5-44ca-896f-ff20c137b557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:15.532490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:15.533371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:15.535770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:15.535770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:15.535770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:15.535770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 333/666] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:15.535770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea62944-e0f5-44ca-896f-ff20c137b557 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.559827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:15.632024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:15.640804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea62944-e0f5-44ca-896f-ff20c137b557 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.648377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c984177-5e43-4c42-bb60-5160f3d2098f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.674538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb1640-4acf-4911-a3e8-da76808bf43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.675116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb1640-4acf-4911-a3e8-da76808bf43b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.675396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb1640-4acf-4911-a3e8-da76808bf43b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.675674 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb1640-4acf-4911-a3e8-da76808bf43b None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.676324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb1640-4acf-4911-a3e8-da76808bf43b None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:15.676620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb1640-4acf-4911-a3e8-da76808bf43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:15.678005 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb1640-4acf-4911-a3e8-da76808bf43b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:15.690821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 334/667] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:15.691169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a6fbf-d9aa-412d-8d21-ae37e44efb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.691296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb1640-4acf-4911-a3e8-da76808bf43b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.691382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a6fbf-d9aa-412d-8d21-ae37e44efb29 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.691382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a6fbf-d9aa-412d-8d21-ae37e44efb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.691382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a6fbf-d9aa-412d-8d21-ae37e44efb29 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.691382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a6fbf-d9aa-412d-8d21-ae37e44efb29 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:15.691382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a6fbf-d9aa-412d-8d21-ae37e44efb29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:15.691382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a6fbf-d9aa-412d-8d21-ae37e44efb29 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:15.709132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 334/668] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:15.709856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a6fbf-d9aa-412d-8d21-ae37e44efb29 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.779856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:15.799424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb1640-4acf-4911-a3e8-da76808bf43b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.814702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a6fbf-d9aa-412d-8d21-ae37e44efb29 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.832873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451fb7-ef62-41be-9d74-6ba5238ce8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.833468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451fb7-ef62-41be-9d74-6ba5238ce8bb None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.833736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451fb7-ef62-41be-9d74-6ba5238ce8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.834022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451fb7-ef62-41be-9d74-6ba5238ce8bb None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.834681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451fb7-ef62-41be-9d74-6ba5238ce8bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:15.834996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451fb7-ef62-41be-9d74-6ba5238ce8bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:15.836955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451fb7-ef62-41be-9d74-6ba5238ce8bb None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:15.848644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 335/669] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:15.849383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451fb7-ef62-41be-9d74-6ba5238ce8bb None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.858073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d7a57-7e7e-4efd-bf4a-5967259b564b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.858981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d7a57-7e7e-4efd-bf4a-5967259b564b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.859496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d7a57-7e7e-4efd-bf4a-5967259b564b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.859861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d7a57-7e7e-4efd-bf4a-5967259b564b None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:15.860513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d7a57-7e7e-4efd-bf4a-5967259b564b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:15.860815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d7a57-7e7e-4efd-bf4a-5967259b564b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:15.862158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d7a57-7e7e-4efd-bf4a-5967259b564b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:15.878167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 335/670] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:15.878167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d7a57-7e7e-4efd-bf4a-5967259b564b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:15.882474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:15.957907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451fb7-ef62-41be-9d74-6ba5238ce8bb None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.988629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d7a57-7e7e-4efd-bf4a-5967259b564b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:15.996178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1369cd-dbee-4fd1-8bc6-753078850fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:15.997146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1369cd-dbee-4fd1-8bc6-753078850fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:15.997146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1369cd-dbee-4fd1-8bc6-753078850fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:15.997329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1369cd-dbee-4fd1-8bc6-753078850fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.007973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1369cd-dbee-4fd1-8bc6-753078850fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.009419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1369cd-dbee-4fd1-8bc6-753078850fe5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.011299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:16.025810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be983c-95f0-47fa-82f2-0668ad62da43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-4a1369cd-dbee-4fd1-8bc6-753078850fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:16.026332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:16.032113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:16.032983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:16.032983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:16.032983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:16.033162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be983c-95f0-47fa-82f2-0668ad62da43 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:16.033162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be983c-95f0-47fa-82f2-0668ad62da43 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:16.033162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be983c-95f0-47fa-82f2-0668ad62da43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.033361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 336/671] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:16.033361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1369cd-dbee-4fd1-8bc6-753078850fe5 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:16.107880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be983c-95f0-47fa-82f2-0668ad62da43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.109336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be983c-95f0-47fa-82f2-0668ad62da43 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.112375 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:16.220278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 336/672] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:16.225255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be983c-95f0-47fa-82f2-0668ad62da43 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:16.253205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1369cd-dbee-4fd1-8bc6-753078850fe5 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:16.288165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c9577-eb2a-48a9-8516-91b22cdcc39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:16.289407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c9577-eb2a-48a9-8516-91b22cdcc39d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:16.289529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c9577-eb2a-48a9-8516-91b22cdcc39d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:16.289792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c9577-eb2a-48a9-8516-91b22cdcc39d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.290503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c9577-eb2a-48a9-8516-91b22cdcc39d None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=904f7f036565419b9fdecc5e47acb45c) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:16.290804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c9577-eb2a-48a9-8516-91b22cdcc39d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.292248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c9577-eb2a-48a9-8516-91b22cdcc39d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.297898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 337/673] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/role_assignments?user.id=904f7f036565419b9fdecc5e47acb45c&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:16.298653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c9577-eb2a-48a9-8516-91b22cdcc39d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:16.344957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be983c-95f0-47fa-82f2-0668ad62da43 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:16.347583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:16.374812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:16.387334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c399e-d70f-465e-abd2-3f4c7cefbfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:16.388478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c399e-d70f-465e-abd2-3f4c7cefbfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:16.388814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c399e-d70f-465e-abd2-3f4c7cefbfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:16.389153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c399e-d70f-465e-abd2-3f4c7cefbfc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.400711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c399e-d70f-465e-abd2-3f4c7cefbfc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.402499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c399e-d70f-465e-abd2-3f4c7cefbfc1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0d8c399e-d70f-465e-abd2-3f4c7cefbfc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:16.423839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:16.424889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:16.425786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:16.425786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:16.425786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:16.425786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:16.425786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:16.425786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:16.425786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:16.425786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:16.432791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c9577-eb2a-48a9-8516-91b22cdcc39d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:16.433568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 337/674] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:16.434710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c399e-d70f-465e-abd2-3f4c7cefbfc1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:16.473935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7b417-6ef6-4aba-b473-57f9ac559907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:16.476693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7b417-6ef6-4aba-b473-57f9ac559907 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:16.477437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7b417-6ef6-4aba-b473-57f9ac559907 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:16.477938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7b417-6ef6-4aba-b473-57f9ac559907 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.478867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7b417-6ef6-4aba-b473-57f9ac559907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:16.479397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7b417-6ef6-4aba-b473-57f9ac559907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.481063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7b417-6ef6-4aba-b473-57f9ac559907 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.498467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 338/675] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:16.499698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7b417-6ef6-4aba-b473-57f9ac559907 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:16.556923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c399e-d70f-465e-abd2-3f4c7cefbfc1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:16.601904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafd726-d1b5-4b2a-9aaa-659f128e740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:16.602544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafd726-d1b5-4b2a-9aaa-659f128e740a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:16.602828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafd726-d1b5-4b2a-9aaa-659f128e740a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:16.603131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafd726-d1b5-4b2a-9aaa-659f128e740a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.615786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafd726-d1b5-4b2a-9aaa-659f128e740a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.617255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafd726-d1b5-4b2a-9aaa-659f128e740a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0eafd726-d1b5-4b2a-9aaa-659f128e740a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:16.633468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:16.634478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:16.635422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:16.635422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:16.635422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:16.635422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 338/676] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:16.636340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafd726-d1b5-4b2a-9aaa-659f128e740a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:16.637978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7b417-6ef6-4aba-b473-57f9ac559907 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:16.658177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:16.659054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:16.683886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89ffa4-d377-4c15-a3b9-02a3501305e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:16.684608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89ffa4-d377-4c15-a3b9-02a3501305e2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:16.684954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89ffa4-d377-4c15-a3b9-02a3501305e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:16.685299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89ffa4-d377-4c15-a3b9-02a3501305e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.686056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89ffa4-d377-4c15-a3b9-02a3501305e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:16.686428 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89ffa4-d377-4c15-a3b9-02a3501305e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.688549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89ffa4-d377-4c15-a3b9-02a3501305e2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.705027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 339/677] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:16.706378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89ffa4-d377-4c15-a3b9-02a3501305e2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:16.754478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafd726-d1b5-4b2a-9aaa-659f128e740a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:16.800247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87f411-4eb9-4e72-9e9d-bf7aa44d610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:16.801260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87f411-4eb9-4e72-9e9d-bf7aa44d610b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:16.801616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87f411-4eb9-4e72-9e9d-bf7aa44d610b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:16.802014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87f411-4eb9-4e72-9e9d-bf7aa44d610b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.813021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87f411-4eb9-4e72-9e9d-bf7aa44d610b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.814847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87f411-4eb9-4e72-9e9d-bf7aa44d610b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-bb87f411-4eb9-4e72-9e9d-bf7aa44d610b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:16.831315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:16.832414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:16.833854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:16.835029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 339/678] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:16.836558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87f411-4eb9-4e72-9e9d-bf7aa44d610b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:16.844279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89ffa4-d377-4c15-a3b9-02a3501305e2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:16.884077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e4781-12e2-4344-9c71-bbab05ebd4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:16.884819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4781-12e2-4344-9c71-bbab05ebd4ab None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:16.885168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4781-12e2-4344-9c71-bbab05ebd4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:16.885520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4781-12e2-4344-9c71-bbab05ebd4ab None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.886328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e4781-12e2-4344-9c71-bbab05ebd4ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:16.886696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e4781-12e2-4344-9c71-bbab05ebd4ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.888179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e4781-12e2-4344-9c71-bbab05ebd4ab None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.902057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 340/679] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:16.903210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e4781-12e2-4344-9c71-bbab05ebd4ab None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:16.919473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:16.943213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87f411-4eb9-4e72-9e9d-bf7aa44d610b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:16.948152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:16.981023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a16e76-084d-4050-a36d-c17f84d12adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:16.981956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a16e76-084d-4050-a36d-c17f84d12adc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:16.982426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a16e76-084d-4050-a36d-c17f84d12adc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:16.982882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a16e76-084d-4050-a36d-c17f84d12adc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:16.983803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a16e76-084d-4050-a36d-c17f84d12adc None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=f1e3ba8d0c7d4f8190cd0b5b057f5638) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:16.984277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a16e76-084d-4050-a36d-c17f84d12adc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:16.986275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a16e76-084d-4050-a36d-c17f84d12adc None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:16.993542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 340/680] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/role_assignments?user.id=f1e3ba8d0c7d4f8190cd0b5b057f5638&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:16.994163 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a16e76-084d-4050-a36d-c17f84d12adc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:17.021099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e4781-12e2-4344-9c71-bbab05ebd4ab None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:17.063702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee045aba-53fd-4ba3-9e31-23ce2d752a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:17.064359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee045aba-53fd-4ba3-9e31-23ce2d752a35 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.064637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee045aba-53fd-4ba3-9e31-23ce2d752a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.064908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee045aba-53fd-4ba3-9e31-23ce2d752a35 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.065728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee045aba-53fd-4ba3-9e31-23ce2d752a35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:17.066034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee045aba-53fd-4ba3-9e31-23ce2d752a35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:17.067589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee045aba-53fd-4ba3-9e31-23ce2d752a35 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:17.093190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 341/681] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:17.094643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fcc80-a452-41db-8630-5d5b82e2a00b None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.094996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fcc80-a452-41db-8630-5d5b82e2a00b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.095293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fcc80-a452-41db-8630-5d5b82e2a00b None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.096902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 342/682] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:17.097646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fcc80-a452-41db-8630-5d5b82e2a00b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:17.136347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a16e76-084d-4050-a36d-c17f84d12adc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:17.178967 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c4844-e763-4bff-90c3-d65bffd42b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:17.179693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c4844-e763-4bff-90c3-d65bffd42b75 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.179970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c4844-e763-4bff-90c3-d65bffd42b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.180239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c4844-e763-4bff-90c3-d65bffd42b75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.188636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c4844-e763-4bff-90c3-d65bffd42b75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:17.190061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c4844-e763-4bff-90c3-d65bffd42b75 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:17.194581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c98c4844-e763-4bff-90c3-d65bffd42b75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:17.205671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:17.206591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 341/683] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c547489-fd87-466e-a3ee-c48c641fefd5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.208283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c547489-fd87-466e-a3ee-c48c641fefd5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.208874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c547489-fd87-466e-a3ee-c48c641fefd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.223511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:17.229524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5c547489-fd87-466e-a3ee-c48c641fefd5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:17.230718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5c547489-fd87-466e-a3ee-c48c641fefd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:17.231910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fcc80-a452-41db-8630-5d5b82e2a00b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:17.270171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e2a52-55ff-44f8-b0e5-1c617bf8abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:17.270804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e2a52-55ff-44f8-b0e5-1c617bf8abf5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.271165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e2a52-55ff-44f8-b0e5-1c617bf8abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.271380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e2a52-55ff-44f8-b0e5-1c617bf8abf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.323036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 342/684] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:17.330047 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c547489-fd87-466e-a3ee-c48c641fefd5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:17.354446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e2a52-55ff-44f8-b0e5-1c617bf8abf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:17.356088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e2a52-55ff-44f8-b0e5-1c617bf8abf5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:17.461155 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:17.473302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 343/685] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:17.474511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e2a52-55ff-44f8-b0e5-1c617bf8abf5 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:17.482479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c547489-fd87-466e-a3ee-c48c641fefd5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:17.519684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:17.527413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e07bf0-b4db-4f64-a712-12ffd880884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:17.528195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e07bf0-b4db-4f64-a712-12ffd880884d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.528615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e07bf0-b4db-4f64-a712-12ffd880884d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.529040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e07bf0-b4db-4f64-a712-12ffd880884d None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.529872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e07bf0-b4db-4f64-a712-12ffd880884d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:17.530314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e07bf0-b4db-4f64-a712-12ffd880884d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:17.531902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e07bf0-b4db-4f64-a712-12ffd880884d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:17.544580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 343/686] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:17.545699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e07bf0-b4db-4f64-a712-12ffd880884d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:17.581305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e2a52-55ff-44f8-b0e5-1c617bf8abf5 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:17.618187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc5fe-db7d-4eae-83c1-d516ea131aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:17.619674 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc5fe-db7d-4eae-83c1-d516ea131aab None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.619949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc5fe-db7d-4eae-83c1-d516ea131aab None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.620216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc5fe-db7d-4eae-83c1-d516ea131aab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.636643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cc5fe-db7d-4eae-83c1-d516ea131aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:17.638819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cc5fe-db7d-4eae-83c1-d516ea131aab None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:17.653619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e07bf0-b4db-4f64-a712-12ffd880884d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-658cc5fe-db7d-4eae-83c1-d516ea131aab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:17.659553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:17.660481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:17.661372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:17.661372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:17.661372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:17.663572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 344/687] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:17.664507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc5fe-db7d-4eae-83c1-d516ea131aab None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:17.696140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962253e-0384-4d6b-9bbd-2012cd1306df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:17.696955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962253e-0384-4d6b-9bbd-2012cd1306df None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.697346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962253e-0384-4d6b-9bbd-2012cd1306df None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.697711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962253e-0384-4d6b-9bbd-2012cd1306df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.708993 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962253e-0384-4d6b-9bbd-2012cd1306df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:17.710761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962253e-0384-4d6b-9bbd-2012cd1306df None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f962253e-0384-4d6b-9bbd-2012cd1306df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:17.726983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:17.728176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:17.729305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:17.729305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:17.729305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:17.735153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 344/688] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:17.735153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962253e-0384-4d6b-9bbd-2012cd1306df None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:17.740126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:17.783654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:17.809469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc5fe-db7d-4eae-83c1-d516ea131aab None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:17.848103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522af561-2909-4223-be37-43ca3a27a8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:17.848724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522af561-2909-4223-be37-43ca3a27a8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.848995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522af561-2909-4223-be37-43ca3a27a8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.849257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522af561-2909-4223-be37-43ca3a27a8d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.850064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522af561-2909-4223-be37-43ca3a27a8d0 None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=ab9b7576a3834862af6abb7e0416f158) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:17.850370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522af561-2909-4223-be37-43ca3a27a8d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:17.852438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522af561-2909-4223-be37-43ca3a27a8d0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:17.854972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962253e-0384-4d6b-9bbd-2012cd1306df None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:17.858852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 345/689] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:17 2022] GET /identity/v3/role_assignments?user.id=ab9b7576a3834862af6abb7e0416f158&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:17.859599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522af561-2909-4223-be37-43ca3a27a8d0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:17.902526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618bb16-375e-400e-8f9a-31f1f1607adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:17.903544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618bb16-375e-400e-8f9a-31f1f1607adf None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:17.904029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618bb16-375e-400e-8f9a-31f1f1607adf None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:17.905036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618bb16-375e-400e-8f9a-31f1f1607adf None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:17.905899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618bb16-375e-400e-8f9a-31f1f1607adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:17.906414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618bb16-375e-400e-8f9a-31f1f1607adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:17.908922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618bb16-375e-400e-8f9a-31f1f1607adf None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:17.924742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 345/690] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:17.930670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618bb16-375e-400e-8f9a-31f1f1607adf None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:17.985972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522af561-2909-4223-be37-43ca3a27a8d0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:18.004703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:18.057909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c4a74-f05c-4a33-a265-69b44cfbac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:18.059326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c4a74-f05c-4a33-a265-69b44cfbac9e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:18.059812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c4a74-f05c-4a33-a265-69b44cfbac9e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:18.060279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c4a74-f05c-4a33-a265-69b44cfbac9e None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:18.061120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c4a74-f05c-4a33-a265-69b44cfbac9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:18.063213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c4a74-f05c-4a33-a265-69b44cfbac9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:18.064824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c4a74-f05c-4a33-a265-69b44cfbac9e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:18.070999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618bb16-375e-400e-8f9a-31f1f1607adf None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:18.089649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:18.095273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 346/691] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:18.102804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c4a74-f05c-4a33-a265-69b44cfbac9e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:18.138444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee6d74-9646-417e-861d-1229e00e8571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:18.139709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee6d74-9646-417e-861d-1229e00e8571 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:18.140142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee6d74-9646-417e-861d-1229e00e8571 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:18.140556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee6d74-9646-417e-861d-1229e00e8571 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:18.142028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee6d74-9646-417e-861d-1229e00e8571 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:18.143900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee6d74-9646-417e-861d-1229e00e8571 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:18.169395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 346/692] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:51:17 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:18.244925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c4a74-f05c-4a33-a265-69b44cfbac9e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:18.287390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6c7f3-81f5-4465-ac5b-b93b1889538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:18.288040 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6c7f3-81f5-4465-ac5b-b93b1889538f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:18.288316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6c7f3-81f5-4465-ac5b-b93b1889538f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:18.289139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6c7f3-81f5-4465-ac5b-b93b1889538f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:18.299813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6c7f3-81f5-4465-ac5b-b93b1889538f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:18.301620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6c7f3-81f5-4465-ac5b-b93b1889538f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-f3d6c7f3-81f5-4465-ac5b-b93b1889538f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:18.314304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:18.315266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:18.316127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:18.316127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:18.316127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:18.316127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:18.316127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:18.316127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:18.316127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:18.316127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:18.319653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 347/693] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:18.328400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee6d74-9646-417e-861d-1229e00e8571 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:18.562609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee6d74-9646-417e-861d-1229e00e8571 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:18.599142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06416445-2c91-4f0e-b998-98f939fe46c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:18.599674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06416445-2c91-4f0e-b998-98f939fe46c7 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:18.599948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06416445-2c91-4f0e-b998-98f939fe46c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:18.600213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06416445-2c91-4f0e-b998-98f939fe46c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:18.600967 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06416445-2c91-4f0e-b998-98f939fe46c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:18.601305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06416445-2c91-4f0e-b998-98f939fe46c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:18.602723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06416445-2c91-4f0e-b998-98f939fe46c7 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:18.615001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 347/694] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:18.620850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6c7f3-81f5-4465-ac5b-b93b1889538f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:18.663900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:18.737201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6c7f3-81f5-4465-ac5b-b93b1889538f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:18.762739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bef794-ce3d-4982-aedd-7d827d534fce None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:18.763721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bef794-ce3d-4982-aedd-7d827d534fce None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:18.764438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bef794-ce3d-4982-aedd-7d827d534fce None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:18.766813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 348/695] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:18.772448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda9671-1fe7-4835-9da1-d0d80bffad0e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:18.773586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda9671-1fe7-4835-9da1-d0d80bffad0e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:18.774323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda9671-1fe7-4835-9da1-d0d80bffad0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:18.790999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f5e92-2d30-471e-a90b-eb48d8e2932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:18.790999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f5e92-2d30-471e-a90b-eb48d8e2932b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:18.790999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f5e92-2d30-471e-a90b-eb48d8e2932b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:18.790999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f5e92-2d30-471e-a90b-eb48d8e2932b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:18.791420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f5e92-2d30-471e-a90b-eb48d8e2932b None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=af29f3451f654f53ac98239a2b28d128) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:18.791420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f5e92-2d30-471e-a90b-eb48d8e2932b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:18.791420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f5e92-2d30-471e-a90b-eb48d8e2932b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:18.795624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6eda9671-1fe7-4835-9da1-d0d80bffad0e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:18.796835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6eda9671-1fe7-4835-9da1-d0d80bffad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:18.808777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 348/696] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:18 2022] GET /identity/v3/role_assignments?user.id=af29f3451f654f53ac98239a2b28d128&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:18.815560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:18.884656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 349/697] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:18.910714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f5e92-2d30-471e-a90b-eb48d8e2932b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:19.039101 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:19.046265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f5e92-2d30-471e-a90b-eb48d8e2932b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:19.089163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d76d2-b422-443a-ae9d-354c192fc4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:19.089785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d76d2-b422-443a-ae9d-354c192fc4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:19.090055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d76d2-b422-443a-ae9d-354c192fc4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:19.092791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d76d2-b422-443a-ae9d-354c192fc4c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:19.102301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d76d2-b422-443a-ae9d-354c192fc4c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-339d76d2-b422-443a-ae9d-354c192fc4c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:19.107447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:19.108374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:19.109201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 349/698] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:19.116042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda9671-1fe7-4835-9da1-d0d80bffad0e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:19.231046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda9671-1fe7-4835-9da1-d0d80bffad0e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:19.253894 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6075c0f-d5b0-40d7-bf65-e26586e2a716 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:19.254176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6075c0f-d5b0-40d7-bf65-e26586e2a716 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:19.254447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6075c0f-d5b0-40d7-bf65-e26586e2a716 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:19.256030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 350/699] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:19.261123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f8653-e5c2-41d6-a307-06d5fa68c37f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:19.261408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f8653-e5c2-41d6-a307-06d5fa68c37f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:19.261684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f8653-e5c2-41d6-a307-06d5fa68c37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:19.268098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b4e18-29df-42bc-b275-5e1de7659894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:19.268976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b4e18-29df-42bc-b275-5e1de7659894 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:19.269430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b4e18-29df-42bc-b275-5e1de7659894 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:19.269846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b4e18-29df-42bc-b275-5e1de7659894 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:19.270718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b4e18-29df-42bc-b275-5e1de7659894 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:19.271199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b4e18-29df-42bc-b275-5e1de7659894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:19.272698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b4e18-29df-42bc-b275-5e1de7659894 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:19.281948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 350/700] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:19.284843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ba3f8653-e5c2-41d6-a307-06d5fa68c37f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:19.284843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ba3f8653-e5c2-41d6-a307-06d5fa68c37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:19.287233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b4e18-29df-42bc-b275-5e1de7659894 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:19.300722 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:19.395371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 351/701] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:51:19.417352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f8653-e5c2-41d6-a307-06d5fa68c37f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:19.437892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b4e18-29df-42bc-b275-5e1de7659894 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:19.479108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fba575-1741-48be-9922-ac3f21cd3285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:19.480385 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fba575-1741-48be-9922-ac3f21cd3285 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:19.480680 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fba575-1741-48be-9922-ac3f21cd3285 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:19.480953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fba575-1741-48be-9922-ac3f21cd3285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:19.494814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fba575-1741-48be-9922-ac3f21cd3285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:19.496230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fba575-1741-48be-9922-ac3f21cd3285 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-43fba575-1741-48be-9922-ac3f21cd3285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:19.513737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:19.514715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:19.515645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:19.515645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:19.515645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:19.515645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 351/702] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:19.521083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fba575-1741-48be-9922-ac3f21cd3285 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:19.590626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:19.635391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fba575-1741-48be-9922-ac3f21cd3285 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:19.640261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:19.674638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f8653-e5c2-41d6-a307-06d5fa68c37f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:19.677566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9a6a6-4ba1-4c45-b55b-63ac3fedfab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:19.678218 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9a6a6-4ba1-4c45-b55b-63ac3fedfab5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:19.678495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9a6a6-4ba1-4c45-b55b-63ac3fedfab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:19.678761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9a6a6-4ba1-4c45-b55b-63ac3fedfab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:19.679461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9a6a6-4ba1-4c45-b55b-63ac3fedfab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:19.679755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9a6a6-4ba1-4c45-b55b-63ac3fedfab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:19.682053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9a6a6-4ba1-4c45-b55b-63ac3fedfab5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:19.698129 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 352/703] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:19.704598 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9a6a6-4ba1-4c45-b55b-63ac3fedfab5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:19.742553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c504e-c191-4465-8156-4182b42fc9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:19.743211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c504e-c191-4465-8156-4182b42fc9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:19.743735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c504e-c191-4465-8156-4182b42fc9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:19.744003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c504e-c191-4465-8156-4182b42fc9c5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:19.744584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c504e-c191-4465-8156-4182b42fc9c5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:19.745994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c504e-c191-4465-8156-4182b42fc9c5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-0c1c504e-c191-4465-8156-4182b42fc9c5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:19.759132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:19.760147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 09:51:19.760997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:19.761767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 352/704] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Aug 10 09:51:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:19.771125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c504e-c191-4465-8156-4182b42fc9c5 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:19.898705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c504e-c191-4465-8156-4182b42fc9c5 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:19.910560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9a6a6-4ba1-4c45-b55b-63ac3fedfab5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:19.934487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed92d5-5e00-4927-86bc-b5022165c403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:19.935118 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed92d5-5e00-4927-86bc-b5022165c403 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:19.935435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed92d5-5e00-4927-86bc-b5022165c403 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:19.935705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed92d5-5e00-4927-86bc-b5022165c403 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:19.936329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed92d5-5e00-4927-86bc-b5022165c403 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:19.936626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed92d5-5e00-4927-86bc-b5022165c403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:19.937979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed92d5-5e00-4927-86bc-b5022165c403 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:19.943191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 353/705] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 10 09:51:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:19.952060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed92d5-5e00-4927-86bc-b5022165c403 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:19.967311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:19.982252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635621e-dad5-4f8e-a05d-980ef47438d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:19.982894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635621e-dad5-4f8e-a05d-980ef47438d2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:19.983232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635621e-dad5-4f8e-a05d-980ef47438d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:19.983554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635621e-dad5-4f8e-a05d-980ef47438d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:19.991764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635621e-dad5-4f8e-a05d-980ef47438d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:19.992054 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:19.994454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635621e-dad5-4f8e-a05d-980ef47438d2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-7635621e-dad5-4f8e-a05d-980ef47438d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:20.010321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:20.011337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:20.012291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 353/706] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 10 09:51:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:20.018287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635621e-dad5-4f8e-a05d-980ef47438d2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:20.169970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed92d5-5e00-4927-86bc-b5022165c403 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:20.209497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635621e-dad5-4f8e-a05d-980ef47438d2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:20.237366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e249c-c9da-4ea3-8546-3ef902e8e758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:20.237973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e249c-c9da-4ea3-8546-3ef902e8e758 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:20.238296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e249c-c9da-4ea3-8546-3ef902e8e758 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:20.238569 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e249c-c9da-4ea3-8546-3ef902e8e758 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:20.239630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e249c-c9da-4ea3-8546-3ef902e8e758 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:20.239936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e249c-c9da-4ea3-8546-3ef902e8e758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:20.241732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e249c-c9da-4ea3-8546-3ef902e8e758 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:20.248862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 354/707] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 10 09:51:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:20.281444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7d3f4-46ac-4066-b39d-96099fd10f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:20.282763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7d3f4-46ac-4066-b39d-96099fd10f85 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:20.283264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7d3f4-46ac-4066-b39d-96099fd10f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:20.283738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7d3f4-46ac-4066-b39d-96099fd10f85 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:20.284695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7d3f4-46ac-4066-b39d-96099fd10f85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:20.285258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7d3f4-46ac-4066-b39d-96099fd10f85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:20.286932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7d3f4-46ac-4066-b39d-96099fd10f85 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:20.321828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 354/708] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 10 09:51:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:20.327621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e249c-c9da-4ea3-8546-3ef902e8e758 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:20.449707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:20.472665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e249c-c9da-4ea3-8546-3ef902e8e758 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:20.509558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4823-543a-4e81-a4bf-55c6afda59b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:20.510605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4823-543a-4e81-a4bf-55c6afda59b0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:20.511073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4823-543a-4e81-a4bf-55c6afda59b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:20.511518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4823-543a-4e81-a4bf-55c6afda59b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:20.597978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb4823-543a-4e81-a4bf-55c6afda59b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:20.599497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb4823-543a-4e81-a4bf-55c6afda59b0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:20.611031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d5fe9-5254-4e80-9452-9a06be706b48 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:20.611719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d5fe9-5254-4e80-9452-9a06be706b48 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:20.612197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d5fe9-5254-4e80-9452-9a06be706b48 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:20.613970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 355/709] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:20.622834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4d8fc-30d5-4db1-8171-22e81629ff28 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:20.623380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4d8fc-30d5-4db1-8171-22e81629ff28 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:20.623874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4d8fc-30d5-4db1-8171-22e81629ff28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:20.666854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d7e4d8fc-30d5-4db1-8171-22e81629ff28 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:20.668449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d7e4d8fc-30d5-4db1-8171-22e81629ff28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:20.699411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:20.706369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 355/710] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:20.712247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4823-543a-4e81-a4bf-55c6afda59b0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:20.765518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:20.818197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4823-543a-4e81-a4bf-55c6afda59b0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:20.856717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 356/711] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:20.863251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b637f69-6d35-4427-a7e5-d29787462c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:20.863946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b637f69-6d35-4427-a7e5-d29787462c93 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:20.864223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b637f69-6d35-4427-a7e5-d29787462c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:20.864494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b637f69-6d35-4427-a7e5-d29787462c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:20.875866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b637f69-6d35-4427-a7e5-d29787462c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:20.877533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b637f69-6d35-4427-a7e5-d29787462c93 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:20.884497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73787273-9ab3-4c40-8d1a-0c00c4e31f1d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:20.884778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73787273-9ab3-4c40-8d1a-0c00c4e31f1d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:20.885047 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73787273-9ab3-4c40-8d1a-0c00c4e31f1d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:20.886376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 357/712] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:20.888280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73787273-9ab3-4c40-8d1a-0c00c4e31f1d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-6b637f69-6d35-4427-a7e5-d29787462c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:20.892814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:20.893780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:20.894697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:20.894697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:20.894697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:20.894697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 356/713] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:20.895712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b149b-fbb1-45f5-b14c-ce86b33a5da1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:20.895991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b149b-fbb1-45f5-b14c-ce86b33a5da1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:20.896258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b149b-fbb1-45f5-b14c-ce86b33a5da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:20.915619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a42b149b-fbb1-45f5-b14c-ce86b33a5da1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:20.916797 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a42b149b-fbb1-45f5-b14c-ce86b33a5da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:20.964680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:20.996845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 357/714] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 09:51:20.997704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b149b-fbb1-45f5-b14c-ce86b33a5da1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.015640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73787273-9ab3-4c40-8d1a-0c00c4e31f1d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.060381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5ec0a-7a7c-4857-9761-4f98654131a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.061268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5ec0a-7a7c-4857-9761-4f98654131a3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.061544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5ec0a-7a7c-4857-9761-4f98654131a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.061812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5ec0a-7a7c-4857-9761-4f98654131a3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.062545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5ec0a-7a7c-4857-9761-4f98654131a3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:21.064494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5ec0a-7a7c-4857-9761-4f98654131a3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-97a5ec0a-7a7c-4857-9761-4f98654131a3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:21.077404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:21.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:21.079358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 358/715] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:20 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:21.080456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5ec0a-7a7c-4857-9761-4f98654131a3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.109676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b149b-fbb1-45f5-b14c-ce86b33a5da1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.119006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:21.150638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51245fb5-f430-4419-b2c3-fe14369528cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.151685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51245fb5-f430-4419-b2c3-fe14369528cd None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.152061 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51245fb5-f430-4419-b2c3-fe14369528cd None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.152964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51245fb5-f430-4419-b2c3-fe14369528cd None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.153851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51245fb5-f430-4419-b2c3-fe14369528cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:21.154306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51245fb5-f430-4419-b2c3-fe14369528cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:21.156885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51245fb5-f430-4419-b2c3-fe14369528cd None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:21.172796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 358/716] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:21.173754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51245fb5-f430-4419-b2c3-fe14369528cd None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.188157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5ec0a-7a7c-4857-9761-4f98654131a3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.207181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:21.224623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ded9b-dbcb-4e79-b01c-f7d8de3cec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.225350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ded9b-dbcb-4e79-b01c-f7d8de3cec17 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.225806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ded9b-dbcb-4e79-b01c-f7d8de3cec17 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.226796 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ded9b-dbcb-4e79-b01c-f7d8de3cec17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.237850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ded9b-dbcb-4e79-b01c-f7d8de3cec17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ba3ded9b-dbcb-4e79-b01c-f7d8de3cec17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:21.242889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:21.244814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:21.246763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:21.254572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 359/717] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:21.254572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ded9b-dbcb-4e79-b01c-f7d8de3cec17 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.309446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51245fb5-f430-4419-b2c3-fe14369528cd None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.355076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b0e83-141b-4ebd-ad4a-87b74a5396fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.356229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b0e83-141b-4ebd-ad4a-87b74a5396fe None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.356655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b0e83-141b-4ebd-ad4a-87b74a5396fe None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.357319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b0e83-141b-4ebd-ad4a-87b74a5396fe None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.358503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b0e83-141b-4ebd-ad4a-87b74a5396fe None admin] RBAC: Adding query filter params (name=network) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:21.359244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b0e83-141b-4ebd-ad4a-87b74a5396fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:21.360851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:21.362663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b0e83-141b-4ebd-ad4a-87b74a5396fe None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:21.371187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 359/718] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:21.371328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ded9b-dbcb-4e79-b01c-f7d8de3cec17 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.372342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b0e83-141b-4ebd-ad4a-87b74a5396fe None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.410560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876912d1-d2be-4f88-a1f1-85fde7bba245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.411386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876912d1-d2be-4f88-a1f1-85fde7bba245 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.411671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876912d1-d2be-4f88-a1f1-85fde7bba245 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.411943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876912d1-d2be-4f88-a1f1-85fde7bba245 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.422110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876912d1-d2be-4f88-a1f1-85fde7bba245 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:21.423742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876912d1-d2be-4f88-a1f1-85fde7bba245 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-876912d1-d2be-4f88-a1f1-85fde7bba245 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:21.435842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:21.436707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:21.437577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:21.437577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:21.437577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:21.437577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:21.437577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:21.437577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:21.437577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:21.437577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:21.437577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 360/719] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:21.438476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876912d1-d2be-4f88-a1f1-85fde7bba245 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.467730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:21.486497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b0e83-141b-4ebd-ad4a-87b74a5396fe None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.522309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9d592-1f79-41d5-944d-c694a8b0eec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.523614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9d592-1f79-41d5-944d-c694a8b0eec0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.523895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9d592-1f79-41d5-944d-c694a8b0eec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.524164 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9d592-1f79-41d5-944d-c694a8b0eec0 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.524842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9d592-1f79-41d5-944d-c694a8b0eec0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:21.525114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9d592-1f79-41d5-944d-c694a8b0eec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:21.526456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9d592-1f79-41d5-944d-c694a8b0eec0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:21.534189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 360/720] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:21.534979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9d592-1f79-41d5-944d-c694a8b0eec0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.550390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876912d1-d2be-4f88-a1f1-85fde7bba245 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.588553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cae586-9e21-41af-9286-4936b42df378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.589157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cae586-9e21-41af-9286-4936b42df378 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.589435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cae586-9e21-41af-9286-4936b42df378 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.589704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cae586-9e21-41af-9286-4936b42df378 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.590330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cae586-9e21-41af-9286-4936b42df378 None admin] RBAC: Adding query filter params (type=network) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:21.590630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cae586-9e21-41af-9286-4936b42df378 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:21.592006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cae586-9e21-41af-9286-4936b42df378 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:21.597569 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 361/721] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:21.598707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813bb8-361d-44a3-b6ee-ab57f26bfe4e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.599017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813bb8-361d-44a3-b6ee-ab57f26bfe4e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.599294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813bb8-361d-44a3-b6ee-ab57f26bfe4e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.600665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 362/722] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:21.605119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813bb8-361d-44a3-b6ee-ab57f26bfe4e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.631291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:21.656019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9d592-1f79-41d5-944d-c694a8b0eec0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.698768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb17ee-e80b-45a5-a6e2-df5203457806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.700024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb17ee-e80b-45a5-a6e2-df5203457806 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.700547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb17ee-e80b-45a5-a6e2-df5203457806 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.700818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb17ee-e80b-45a5-a6e2-df5203457806 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.701536 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb17ee-e80b-45a5-a6e2-df5203457806 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:21.701835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb17ee-e80b-45a5-a6e2-df5203457806 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:21.703404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb17ee-e80b-45a5-a6e2-df5203457806 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:21.722456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 361/723] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:21.724066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc025d0b-c0e4-4697-8c56-a65041bf3844 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.724345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc025d0b-c0e4-4697-8c56-a65041bf3844 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.724610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc025d0b-c0e4-4697-8c56-a65041bf3844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.735028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813bb8-361d-44a3-b6ee-ab57f26bfe4e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.738681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:21.747179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-fc025d0b-c0e4-4697-8c56-a65041bf3844 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:21.749293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-fc025d0b-c0e4-4697-8c56-a65041bf3844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:21.777322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ee313-0440-4f0c-96c6-421fb63e2467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.778578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ee313-0440-4f0c-96c6-421fb63e2467 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.779039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ee313-0440-4f0c-96c6-421fb63e2467 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.779671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ee313-0440-4f0c-96c6-421fb63e2467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.798454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ee313-0440-4f0c-96c6-421fb63e2467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:21.799990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ee313-0440-4f0c-96c6-421fb63e2467 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-168ee313-0440-4f0c-96c6-421fb63e2467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:21.815883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:21.816838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:21.817729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:21.817729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:21.817729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:21.817729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 363/724] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:21.817947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ee313-0440-4f0c-96c6-421fb63e2467 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.832982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 362/725] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:21.834561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc025d0b-c0e4-4697-8c56-a65041bf3844 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:21.895340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:21.929449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ee313-0440-4f0c-96c6-421fb63e2467 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.960065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc025d0b-c0e4-4697-8c56-a65041bf3844 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:21.965690 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79b06b-7358-4628-b697-c60ff3593bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:21.966304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79b06b-7358-4628-b697-c60ff3593bcb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:21.966580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79b06b-7358-4628-b697-c60ff3593bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:21.966850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79b06b-7358-4628-b697-c60ff3593bcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:21.967659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79b06b-7358-4628-b697-c60ff3593bcb None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=904f7f036565419b9fdecc5e47acb45c) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:21.967972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79b06b-7358-4628-b697-c60ff3593bcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:21.969400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79b06b-7358-4628-b697-c60ff3593bcb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:21.975688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 364/726] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/role_assignments?user.id=904f7f036565419b9fdecc5e47acb45c&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=8417343e9ef64991bcf51bb9692da102 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:21.977508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79b06b-7358-4628-b697-c60ff3593bcb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:22.006636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4acb08-8f82-4b7b-ae8b-86526e03403a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:22.007909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4acb08-8f82-4b7b-ae8b-86526e03403a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.008412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4acb08-8f82-4b7b-ae8b-86526e03403a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.009288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4acb08-8f82-4b7b-ae8b-86526e03403a None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.010454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4acb08-8f82-4b7b-ae8b-86526e03403a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:22.011004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4acb08-8f82-4b7b-ae8b-86526e03403a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:22.013474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4acb08-8f82-4b7b-ae8b-86526e03403a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:22.017373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:22.030648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 363/727] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:22.037592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4acb08-8f82-4b7b-ae8b-86526e03403a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:22.150063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79b06b-7358-4628-b697-c60ff3593bcb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:22.162425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4acb08-8f82-4b7b-ae8b-86526e03403a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:22.203692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce064a06-484e-4bb0-bf46-591fcb8158bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:22.204360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce064a06-484e-4bb0-bf46-591fcb8158bb None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.204594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce064a06-484e-4bb0-bf46-591fcb8158bb None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.204871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce064a06-484e-4bb0-bf46-591fcb8158bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.215054 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce064a06-484e-4bb0-bf46-591fcb8158bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:22.216517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce064a06-484e-4bb0-bf46-591fcb8158bb None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-ce064a06-484e-4bb0-bf46-591fcb8158bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:22.229355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:22.230339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:22.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:22.232146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 364/728] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:51:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:22.236877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce064a06-484e-4bb0-bf46-591fcb8158bb None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:22.241970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6ef50-4919-407d-a08f-864274ed1278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:22.242894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6ef50-4919-407d-a08f-864274ed1278 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.243689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6ef50-4919-407d-a08f-864274ed1278 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.244160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6ef50-4919-407d-a08f-864274ed1278 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.246897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:22.261212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6ef50-4919-407d-a08f-864274ed1278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f6d6ef50-4919-407d-a08f-864274ed1278 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:22.268637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:22.269593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:22.270432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:22.274336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 365/729] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:22.279929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6ef50-4919-407d-a08f-864274ed1278 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:22.318132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:22.396203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce064a06-484e-4bb0-bf46-591fcb8158bb None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:22.460953 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a332cf-a51c-45ba-bd51-89c950724123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:22.461608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a332cf-a51c-45ba-bd51-89c950724123 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.461881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a332cf-a51c-45ba-bd51-89c950724123 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.462146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a332cf-a51c-45ba-bd51-89c950724123 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.463123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a332cf-a51c-45ba-bd51-89c950724123 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:22.463425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a332cf-a51c-45ba-bd51-89c950724123 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:22.465438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6ef50-4919-407d-a08f-864274ed1278 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:22.468716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a332cf-a51c-45ba-bd51-89c950724123 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:22.504225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 365/730] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 10 09:51:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:22.512199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a332cf-a51c-45ba-bd51-89c950724123 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:22.534211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496a781-21ef-4e44-be0e-b4c7ed9bea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:22.535283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496a781-21ef-4e44-be0e-b4c7ed9bea37 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.536132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496a781-21ef-4e44-be0e-b4c7ed9bea37 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.536731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496a781-21ef-4e44-be0e-b4c7ed9bea37 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.537651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496a781-21ef-4e44-be0e-b4c7ed9bea37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:22.538101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496a781-21ef-4e44-be0e-b4c7ed9bea37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:22.540107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496a781-21ef-4e44-be0e-b4c7ed9bea37 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:22.562415 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 366/731] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:22.570829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496a781-21ef-4e44-be0e-b4c7ed9bea37 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:22.647542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:22.667332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a332cf-a51c-45ba-bd51-89c950724123 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:22.683086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:22.696913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496a781-21ef-4e44-be0e-b4c7ed9bea37 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:22.705105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053440f1-a163-435f-824b-7cc08fe1b1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:22.705722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053440f1-a163-435f-824b-7cc08fe1b1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.705998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053440f1-a163-435f-824b-7cc08fe1b1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.706268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053440f1-a163-435f-824b-7cc08fe1b1b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.735426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d35ce5-96fe-439c-9780-893680376640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:22.737006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d35ce5-96fe-439c-9780-893680376640 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.737736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d35ce5-96fe-439c-9780-893680376640 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.738213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d35ce5-96fe-439c-9780-893680376640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.753217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d35ce5-96fe-439c-9780-893680376640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:22.755244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d35ce5-96fe-439c-9780-893680376640 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f6d35ce5-96fe-439c-9780-893680376640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:22.775034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:22.775985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:22.776995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:22.776995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:22.776995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:22.779627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 367/732] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:22.781212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053440f1-a163-435f-824b-7cc08fe1b1b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:22.781520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2f79d-52fd-4fdb-a941-4ade4f09c2ff None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.781952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2f79d-52fd-4fdb-a941-4ade4f09c2ff None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.782374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2f79d-52fd-4fdb-a941-4ade4f09c2ff None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.783086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053440f1-a163-435f-824b-7cc08fe1b1b2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:22.784261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 368/733] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:22.785123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2f79d-52fd-4fdb-a941-4ade4f09c2ff None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:22.903871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 366/734] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:22.905469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39136b44-b221-468d-971c-c761d61dc433 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.905866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39136b44-b221-468d-971c-c761d61dc433 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.906242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39136b44-b221-468d-971c-c761d61dc433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.925247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-39136b44-b221-468d-971c-c761d61dc433 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:22.926506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-39136b44-b221-468d-971c-c761d61dc433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:22.933958 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2f79d-52fd-4fdb-a941-4ade4f09c2ff None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:22.938296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:22.949544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:22.984435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ee9b8-aa3f-496c-b083-25d02db065f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:22.985195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ee9b8-aa3f-496c-b083-25d02db065f0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:22.985574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ee9b8-aa3f-496c-b083-25d02db065f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:22.985949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ee9b8-aa3f-496c-b083-25d02db065f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:22.986698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ee9b8-aa3f-496c-b083-25d02db065f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:22.987303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ee9b8-aa3f-496c-b083-25d02db065f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:22.988613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ee9b8-aa3f-496c-b083-25d02db065f0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:23.001402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 369/735] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:23.002584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ee9b8-aa3f-496c-b083-25d02db065f0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:23.020648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 367/736] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:23.021741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39136b44-b221-468d-971c-c761d61dc433 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:23.135955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39136b44-b221-468d-971c-c761d61dc433 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:23.147769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ee9b8-aa3f-496c-b083-25d02db065f0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:23.173242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217c8f9-6f8b-43c8-9926-7e32d41ee750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:23.174003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217c8f9-6f8b-43c8-9926-7e32d41ee750 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:23.174383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217c8f9-6f8b-43c8-9926-7e32d41ee750 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:23.174770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217c8f9-6f8b-43c8-9926-7e32d41ee750 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:23.183358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217c8f9-6f8b-43c8-9926-7e32d41ee750 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:23.184875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217c8f9-6f8b-43c8-9926-7e32d41ee750 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:23.193361 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447fbb0-bb20-4f37-813f-533f622ef8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:23.194193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447fbb0-bb20-4f37-813f-533f622ef8dc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:23.194640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447fbb0-bb20-4f37-813f-533f622ef8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:23.195118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447fbb0-bb20-4f37-813f-533f622ef8dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-4217c8f9-6f8b-43c8-9926-7e32d41ee750 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:23.197291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:23.198412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:23.199490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:23.200401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 368/737] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:23.201448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217c8f9-6f8b-43c8-9926-7e32d41ee750 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:23.205979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:23.213307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447fbb0-bb20-4f37-813f-533f622ef8dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:23.215368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447fbb0-bb20-4f37-813f-533f622ef8dc None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-4447fbb0-bb20-4f37-813f-533f622ef8dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:23.234809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:23.235835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:23.236751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:23.236751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:23.236751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:23.239088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 370/738] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:23.239878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447fbb0-bb20-4f37-813f-533f622ef8dc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:23.269628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:23.319644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217c8f9-6f8b-43c8-9926-7e32d41ee750 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:23.356523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0a023-1684-43d0-9856-9f9f7f33eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:23.357136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0a023-1684-43d0-9856-9f9f7f33eab2 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:23.357417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0a023-1684-43d0-9856-9f9f7f33eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:23.357693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0a023-1684-43d0-9856-9f9f7f33eab2 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:23.358337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0a023-1684-43d0-9856-9f9f7f33eab2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:23.359734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0a023-1684-43d0-9856-9f9f7f33eab2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:23.380165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447fbb0-bb20-4f37-813f-533f622ef8dc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:23.384966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 369/739] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:51:23 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:23.385908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0a023-1684-43d0-9856-9f9f7f33eab2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:23.462645 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:23.467238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff2ea7-a29d-4365-92ae-0b528a0a51e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:23.467748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff2ea7-a29d-4365-92ae-0b528a0a51e6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:23.468752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff2ea7-a29d-4365-92ae-0b528a0a51e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:23.468752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff2ea7-a29d-4365-92ae-0b528a0a51e6 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:23.468919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff2ea7-a29d-4365-92ae-0b528a0a51e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:23.469535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff2ea7-a29d-4365-92ae-0b528a0a51e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:23.470984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff2ea7-a29d-4365-92ae-0b528a0a51e6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:23.509123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 371/740] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 10 09:51:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:23.515087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff2ea7-a29d-4365-92ae-0b528a0a51e6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:23.524903 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0a023-1684-43d0-9856-9f9f7f33eab2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:23.593877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad99621-7687-4879-a5cd-e25093aed38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:23.594548 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad99621-7687-4879-a5cd-e25093aed38e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:23.594832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad99621-7687-4879-a5cd-e25093aed38e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:23.595131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad99621-7687-4879-a5cd-e25093aed38e None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:23.595783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad99621-7687-4879-a5cd-e25093aed38e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:23.596086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad99621-7687-4879-a5cd-e25093aed38e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:23.597469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad99621-7687-4879-a5cd-e25093aed38e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:23.615824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:23.620534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 370/741] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:23.627620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad99621-7687-4879-a5cd-e25093aed38e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:23.669040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff2ea7-a29d-4365-92ae-0b528a0a51e6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:23.713933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5cf81-a58b-4a38-a9d5-a46f849b24df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:23.714467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5cf81-a58b-4a38-a9d5-a46f849b24df None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:23.714747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5cf81-a58b-4a38-a9d5-a46f849b24df None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:23.715064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5cf81-a58b-4a38-a9d5-a46f849b24df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:23.768830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad99621-7687-4879-a5cd-e25093aed38e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:23.793718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5cf81-a58b-4a38-a9d5-a46f849b24df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:23.795445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5cf81-a58b-4a38-a9d5-a46f849b24df None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:23.800125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:23.812703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24ce8d-0652-4ee9-a8e8-72fb428d0390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:23.813334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ce8d-0652-4ee9-a8e8-72fb428d0390 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:23.813609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ce8d-0652-4ee9-a8e8-72fb428d0390 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:23.813880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ce8d-0652-4ee9-a8e8-72fb428d0390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:23.821640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ce8d-0652-4ee9-a8e8-72fb428d0390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:23.823130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ce8d-0652-4ee9-a8e8-72fb428d0390 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-5e24ce8d-0652-4ee9-a8e8-72fb428d0390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:23.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:23.836707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:23.837602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:23.837602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:23.837602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:23.837602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:23.837602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:23.837602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:23.837602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:23.837602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:23.838022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 371/742] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:23.843617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24ce8d-0652-4ee9-a8e8-72fb428d0390 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:23.887405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:23.919080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 372/743] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:23 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 09:51:23.920517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dafd11-3105-4e8f-990c-2d0bdf7a88fa None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:23.920880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dafd11-3105-4e8f-990c-2d0bdf7a88fa None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:23.921221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dafd11-3105-4e8f-990c-2d0bdf7a88fa None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:23.923818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 373/744] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:23.924817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dafd11-3105-4e8f-990c-2d0bdf7a88fa None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:23.986116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24ce8d-0652-4ee9-a8e8-72fb428d0390 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:24.021783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0f4cf-9dc0-47e5-8575-950ded237fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:24.022392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f4cf-9dc0-47e5-8575-950ded237fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.022672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f4cf-9dc0-47e5-8575-950ded237fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.023536 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f4cf-9dc0-47e5-8575-950ded237fe5 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.024235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0f4cf-9dc0-47e5-8575-950ded237fe5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:24.024540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0f4cf-9dc0-47e5-8575-950ded237fe5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:24.025981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0f4cf-9dc0-47e5-8575-950ded237fe5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:24.038994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 372/745] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:24.040410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25f1aa-9826-4369-9e51-a567eb4d0890 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.040704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25f1aa-9826-4369-9e51-a567eb4d0890 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.040978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25f1aa-9826-4369-9e51-a567eb4d0890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.054994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dafd11-3105-4e8f-990c-2d0bdf7a88fa None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:24.060314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-fb25f1aa-9826-4369-9e51-a567eb4d0890 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:24.061509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-fb25f1aa-9826-4369-9e51-a567eb4d0890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:24.102131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51894b51-b088-482f-a312-32f12001d49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:24.103391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51894b51-b088-482f-a312-32f12001d49c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.104050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51894b51-b088-482f-a312-32f12001d49c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.104700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51894b51-b088-482f-a312-32f12001d49c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.105515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:24.122596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51894b51-b088-482f-a312-32f12001d49c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:24.124834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51894b51-b088-482f-a312-32f12001d49c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:24.142490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 373/746] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 09:51:24.142490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25f1aa-9826-4369-9e51-a567eb4d0890 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-51894b51-b088-482f-a312-32f12001d49c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:24.152809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:24.153761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:24.154625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:24.154625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:24.154625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:24.156984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 374/747] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:24.163791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51894b51-b088-482f-a312-32f12001d49c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:24.204552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:24.263811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25f1aa-9826-4369-9e51-a567eb4d0890 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:24.297495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51894b51-b088-482f-a312-32f12001d49c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:24.301006 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a1e20-7063-4898-918b-6e245dba00dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:24.301608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a1e20-7063-4898-918b-6e245dba00dc None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.301884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a1e20-7063-4898-918b-6e245dba00dc None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.302154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a1e20-7063-4898-918b-6e245dba00dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.302875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a1e20-7063-4898-918b-6e245dba00dc None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=36f9577e047a406480fadc4b9f8c6234) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:24.303209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a1e20-7063-4898-918b-6e245dba00dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:24.305195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a1e20-7063-4898-918b-6e245dba00dc None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:24.314992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 374/748] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:24 2022] GET /identity/v3/role_assignments?user.id=36f9577e047a406480fadc4b9f8c6234&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:24.314992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a1e20-7063-4898-918b-6e245dba00dc None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:24.352836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87194081-5db4-4211-9a27-d239b6cfb0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:24.358976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87194081-5db4-4211-9a27-d239b6cfb0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.358976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87194081-5db4-4211-9a27-d239b6cfb0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.358976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87194081-5db4-4211-9a27-d239b6cfb0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.358976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87194081-5db4-4211-9a27-d239b6cfb0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:24.358976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87194081-5db4-4211-9a27-d239b6cfb0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:24.358976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87194081-5db4-4211-9a27-d239b6cfb0b1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:24.359482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:24.382856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 375/749] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:24.385249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc580-1efe-42e1-acf5-45ec4bf199b6 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.385539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc580-1efe-42e1-acf5-45ec4bf199b6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.385821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc580-1efe-42e1-acf5-45ec4bf199b6 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.388643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 376/750] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:24.389708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdc580-1efe-42e1-acf5-45ec4bf199b6 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:24.450413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a1e20-7063-4898-918b-6e245dba00dc None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:24.489336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5f3c8-8298-4d41-9d5c-374ad87f9830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:24.489947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5f3c8-8298-4d41-9d5c-374ad87f9830 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.490224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5f3c8-8298-4d41-9d5c-374ad87f9830 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.490495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5f3c8-8298-4d41-9d5c-374ad87f9830 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.491177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5f3c8-8298-4d41-9d5c-374ad87f9830 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:24.493212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5f3c8-8298-4d41-9d5c-374ad87f9830 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:24.496144 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:24.512434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 375/751] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:51:24 2022] POST /identity/v3/services => generated 240 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:24.513722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66676ad-a4a8-4ed6-a2cd-da1f6dc265a2 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.514153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66676ad-a4a8-4ed6-a2cd-da1f6dc265a2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.514436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66676ad-a4a8-4ed6-a2cd-da1f6dc265a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.533458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdc580-1efe-42e1-acf5-45ec4bf199b6 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:24.553927 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f66676ad-a4a8-4ed6-a2cd-da1f6dc265a2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:24.556124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f66676ad-a4a8-4ed6-a2cd-da1f6dc265a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:24.612536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0950-75c9-4a90-929f-0d700fa78966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:24.613244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0950-75c9-4a90-929f-0d700fa78966 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.613524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0950-75c9-4a90-929f-0d700fa78966 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.613848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0950-75c9-4a90-929f-0d700fa78966 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.623455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe0950-75c9-4a90-929f-0d700fa78966 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:24.625005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe0950-75c9-4a90-929f-0d700fa78966 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-5dfe0950-75c9-4a90-929f-0d700fa78966 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:24.643353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:24.644319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:24.645188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:24.645188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:24.645188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:24.645188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:24.645188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:24.645188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:24.645188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:24.645188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:24.656998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 377/752] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:24.657977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0950-75c9-4a90-929f-0d700fa78966 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:24.699187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 376/753] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:24.722213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66676ad-a4a8-4ed6-a2cd-da1f6dc265a2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:24.755083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:24.780309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0950-75c9-4a90-929f-0d700fa78966 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:24.828045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02338c24-574f-4812-8ecb-c31202854087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:24.828676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02338c24-574f-4812-8ecb-c31202854087 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.828958 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02338c24-574f-4812-8ecb-c31202854087 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.829233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02338c24-574f-4812-8ecb-c31202854087 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.829951 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02338c24-574f-4812-8ecb-c31202854087 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:24.830255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02338c24-574f-4812-8ecb-c31202854087 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:24.831759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02338c24-574f-4812-8ecb-c31202854087 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:24.845768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 378/754] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:24.852229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02338c24-574f-4812-8ecb-c31202854087 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:24.852915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66676ad-a4a8-4ed6-a2cd-da1f6dc265a2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:24.858443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:24.903139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f248ef-6137-4c55-a2f0-6fe322faa6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:24.904655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f248ef-6137-4c55-a2f0-6fe322faa6de None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:24.905129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f248ef-6137-4c55-a2f0-6fe322faa6de None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:24.905590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f248ef-6137-4c55-a2f0-6fe322faa6de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:24.920163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f248ef-6137-4c55-a2f0-6fe322faa6de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-a9f248ef-6137-4c55-a2f0-6fe322faa6de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:24.927292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:24.928227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:24.929609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:24.932691 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 377/755] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:24.938362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f248ef-6137-4c55-a2f0-6fe322faa6de None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:24.979744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02338c24-574f-4812-8ecb-c31202854087 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:25.020624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8aa1d6-b7a3-49da-8d51-238795d4c6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:25.021277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8aa1d6-b7a3-49da-8d51-238795d4c6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:25.021559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8aa1d6-b7a3-49da-8d51-238795d4c6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:25.021883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8aa1d6-b7a3-49da-8d51-238795d4c6f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:25.025081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8aa1d6-b7a3-49da-8d51-238795d4c6f9 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=af29f3451f654f53ac98239a2b28d128) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:25.025081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8aa1d6-b7a3-49da-8d51-238795d4c6f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:25.025081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8aa1d6-b7a3-49da-8d51-238795d4c6f9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:25.031403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 379/756] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:24 2022] GET /identity/v3/role_assignments?user.id=af29f3451f654f53ac98239a2b28d128&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=8417343e9ef64991bcf51bb9692da102 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:25.115540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:25.132849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f248ef-6137-4c55-a2f0-6fe322faa6de None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:25.185161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5031a-c460-435c-925b-610d4b49d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:25.185785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5031a-c460-435c-925b-610d4b49d9cd None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:25.186062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5031a-c460-435c-925b-610d4b49d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:25.186334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5031a-c460-435c-925b-610d4b49d9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:25.187029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5031a-c460-435c-925b-610d4b49d9cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:25.187332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5031a-c460-435c-925b-610d4b49d9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:25.188710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5031a-c460-435c-925b-610d4b49d9cd None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:25.203871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 378/757] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:25.209563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8aa1d6-b7a3-49da-8d51-238795d4c6f9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:25.311724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:25.382970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8aa1d6-b7a3-49da-8d51-238795d4c6f9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:25.464435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e5066-d588-4519-8448-c1cf7e515347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:25.465426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e5066-d588-4519-8448-c1cf7e515347 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:25.465913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e5066-d588-4519-8448-c1cf7e515347 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:25.466406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e5066-d588-4519-8448-c1cf7e515347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:25.482024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e5066-d588-4519-8448-c1cf7e515347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:25.483832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e5066-d588-4519-8448-c1cf7e515347 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f69e5066-d588-4519-8448-c1cf7e515347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:25.506706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:25.508213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:25.509086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:25.509086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:25.509086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:25.511177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 380/758] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:25.517050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5031a-c460-435c-925b-610d4b49d9cd None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:25.666229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5031a-c460-435c-925b-610d4b49d9cd None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:25.717026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23c527-528a-4f4e-9841-0f00069abd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:25.717632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23c527-528a-4f4e-9841-0f00069abd44 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:25.717905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23c527-528a-4f4e-9841-0f00069abd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:25.718172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23c527-528a-4f4e-9841-0f00069abd44 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:25.718803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23c527-528a-4f4e-9841-0f00069abd44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:25.719128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23c527-528a-4f4e-9841-0f00069abd44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:25.720475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23c527-528a-4f4e-9841-0f00069abd44 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:25.735610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 379/759] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:25.744281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e5066-d588-4519-8448-c1cf7e515347 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:25.847705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23db65-88d2-413e-9859-d81567310fe4 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:25.848336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23db65-88d2-413e-9859-d81567310fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:25.848811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23db65-88d2-413e-9859-d81567310fe4 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:25.850637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 380/760] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:25.856281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f854c-1234-4431-b965-9c483df82303 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:25.856784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f854c-1234-4431-b965-9c483df82303 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:25.857274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f854c-1234-4431-b965-9c483df82303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:25.875956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-992f854c-1234-4431-b965-9c483df82303 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:25.877405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-992f854c-1234-4431-b965-9c483df82303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:25.880826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:25.925804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e5066-d588-4519-8448-c1cf7e515347 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:25.957891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 381/761] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:25.972314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:25.987217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f854c-1234-4431-b965-9c483df82303 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:26.010145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15581d3b-4dd0-48bb-89d2-f48b5365b07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:26.011255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15581d3b-4dd0-48bb-89d2-f48b5365b07b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.012612 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15581d3b-4dd0-48bb-89d2-f48b5365b07b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.013859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15581d3b-4dd0-48bb-89d2-f48b5365b07b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.028276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15581d3b-4dd0-48bb-89d2-f48b5365b07b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:26.030667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15581d3b-4dd0-48bb-89d2-f48b5365b07b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-15581d3b-4dd0-48bb-89d2-f48b5365b07b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:26.050839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:26.051823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:26.052763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:26.067836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 381/762] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 10 09:51:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:26.070259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15581d3b-4dd0-48bb-89d2-f48b5365b07b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:26.176686 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f854c-1234-4431-b965-9c483df82303 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:26.235463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144975e-c9e5-46e3-a52f-9bbb57a36894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:26.236272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144975e-c9e5-46e3-a52f-9bbb57a36894 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.236550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144975e-c9e5-46e3-a52f-9bbb57a36894 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.236820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144975e-c9e5-46e3-a52f-9bbb57a36894 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.237396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144975e-c9e5-46e3-a52f-9bbb57a36894 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:26.238866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144975e-c9e5-46e3-a52f-9bbb57a36894 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-3144975e-c9e5-46e3-a52f-9bbb57a36894 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:26.252358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:26.253335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 09:51:26.254160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:26.263297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 382/763] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Aug 10 09:51:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:26.264120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144975e-c9e5-46e3-a52f-9bbb57a36894 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:26.266223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15581d3b-4dd0-48bb-89d2-f48b5365b07b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:26.271524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:26.327154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac758-0b45-4035-a960-744a4eb20b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:26.327760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac758-0b45-4035-a960-744a4eb20b16 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.328037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac758-0b45-4035-a960-744a4eb20b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.328302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac758-0b45-4035-a960-744a4eb20b16 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.332130 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac758-0b45-4035-a960-744a4eb20b16 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:26.332440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac758-0b45-4035-a960-744a4eb20b16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:26.333861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac758-0b45-4035-a960-744a4eb20b16 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:26.376726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 382/764] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 10 09:51:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:26.380114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac758-0b45-4035-a960-744a4eb20b16 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:26.459510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:26.502317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac758-0b45-4035-a960-744a4eb20b16 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:26.510293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144975e-c9e5-46e3-a52f-9bbb57a36894 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:26.537939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf6832-abac-49f3-b676-607cfb54fe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:26.538622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf6832-abac-49f3-b676-607cfb54fe6a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.539020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf6832-abac-49f3-b676-607cfb54fe6a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.539312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf6832-abac-49f3-b676-607cfb54fe6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.550404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf6832-abac-49f3-b676-607cfb54fe6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:26.552033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf6832-abac-49f3-b676-607cfb54fe6a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:26.570290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30907950-d6e9-4f69-90bb-ab77a5db99d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-bbcf6832-abac-49f3-b676-607cfb54fe6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:26.570798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:26.571799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:26.572664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:26.572664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:26.572664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:26.572842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30907950-d6e9-4f69-90bb-ab77a5db99d2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.572842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30907950-d6e9-4f69-90bb-ab77a5db99d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.572842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30907950-d6e9-4f69-90bb-ab77a5db99d2 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.572842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30907950-d6e9-4f69-90bb-ab77a5db99d2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:26.572842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30907950-d6e9-4f69-90bb-ab77a5db99d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:26.578998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30907950-d6e9-4f69-90bb-ab77a5db99d2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:26.579202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 383/765] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:26.579202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27331c75-08c3-4a51-a91f-83a18ed51697 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.579202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27331c75-08c3-4a51-a91f-83a18ed51697 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.579202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27331c75-08c3-4a51-a91f-83a18ed51697 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.579202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 384/766] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:26.580584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27331c75-08c3-4a51-a91f-83a18ed51697 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:26.583003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 383/767] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 10 09:51:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:26.585947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29615684-f04c-4bd2-bf6c-6b23c4f6b70d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.586228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29615684-f04c-4bd2-bf6c-6b23c4f6b70d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.586498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29615684-f04c-4bd2-bf6c-6b23c4f6b70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.604139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-29615684-f04c-4bd2-bf6c-6b23c4f6b70d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:26.605352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-29615684-f04c-4bd2-bf6c-6b23c4f6b70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:26.669382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:26.675765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 384/768] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:26.677066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29615684-f04c-4bd2-bf6c-6b23c4f6b70d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:26.726781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:26.746231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27331c75-08c3-4a51-a91f-83a18ed51697 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:26.796761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d754f4d-4183-4549-ba1e-dd37503a7032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:26.797990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d754f4d-4183-4549-ba1e-dd37503a7032 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.798894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d754f4d-4183-4549-ba1e-dd37503a7032 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.799454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d754f4d-4183-4549-ba1e-dd37503a7032 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.800567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d754f4d-4183-4549-ba1e-dd37503a7032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:26.801313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d754f4d-4183-4549-ba1e-dd37503a7032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:26.803563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d754f4d-4183-4549-ba1e-dd37503a7032 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:26.805166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29615684-f04c-4bd2-bf6c-6b23c4f6b70d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:26.824448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 385/769] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:26.824448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d754f4d-4183-4549-ba1e-dd37503a7032 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:26.856388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e38bf-51be-40cb-96a7-cea72fae61b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:26.857289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e38bf-51be-40cb-96a7-cea72fae61b0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.857958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e38bf-51be-40cb-96a7-cea72fae61b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.858813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e38bf-51be-40cb-96a7-cea72fae61b0 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.859952 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e38bf-51be-40cb-96a7-cea72fae61b0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:26.860405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e38bf-51be-40cb-96a7-cea72fae61b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:26.862249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e38bf-51be-40cb-96a7-cea72fae61b0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:26.870700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 385/770] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 10 09:51:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:26.871940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e38bf-51be-40cb-96a7-cea72fae61b0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:26.934094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d754f4d-4183-4549-ba1e-dd37503a7032 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:26.958068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:26.970153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80048ccc-d3e0-4d5a-b4d5-23d7795b686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:26.971010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80048ccc-d3e0-4d5a-b4d5-23d7795b686d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:26.971391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80048ccc-d3e0-4d5a-b4d5-23d7795b686d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:26.971762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80048ccc-d3e0-4d5a-b4d5-23d7795b686d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:26.984678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80048ccc-d3e0-4d5a-b4d5-23d7795b686d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-80048ccc-d3e0-4d5a-b4d5-23d7795b686d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:26.989971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:26.991096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:26.992736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:26.999460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 386/771] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:26.999460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80048ccc-d3e0-4d5a-b4d5-23d7795b686d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.013198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:27.021023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e38bf-51be-40cb-96a7-cea72fae61b0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.069376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299221c-f945-443a-ae17-9b8665fb4110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.070378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299221c-f945-443a-ae17-9b8665fb4110 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.070841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299221c-f945-443a-ae17-9b8665fb4110 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.071318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299221c-f945-443a-ae17-9b8665fb4110 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.080864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299221c-f945-443a-ae17-9b8665fb4110 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.083048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299221c-f945-443a-ae17-9b8665fb4110 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-b299221c-f945-443a-ae17-9b8665fb4110 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:27.098450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:27.099396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:27.100285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:27.100285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:27.100285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:27.100285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:27.100285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:27.100285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:27.100285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:27.100285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:27.107696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 386/772] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:27.107696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299221c-f945-443a-ae17-9b8665fb4110 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.120384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80048ccc-d3e0-4d5a-b4d5-23d7795b686d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.159271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72507692-fa7a-4fdf-9d97-bd0fb17ce96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.159960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72507692-fa7a-4fdf-9d97-bd0fb17ce96d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.160212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72507692-fa7a-4fdf-9d97-bd0fb17ce96d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.160487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72507692-fa7a-4fdf-9d97-bd0fb17ce96d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.161192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72507692-fa7a-4fdf-9d97-bd0fb17ce96d None admin] RBAC: Adding query filter params (interface=public, service_id=ed9ed16ffbe44f2ba88a5142d2e9c42c, region_id=RegionOne) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:27.161499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72507692-fa7a-4fdf-9d97-bd0fb17ce96d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.162990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72507692-fa7a-4fdf-9d97-bd0fb17ce96d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:27.171662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 387/773] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 10 09:51:26 2022] GET /identity/v3/endpoints?service_id=ed9ed16ffbe44f2ba88a5142d2e9c42c&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:27.176560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72507692-fa7a-4fdf-9d97-bd0fb17ce96d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.211116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299221c-f945-443a-ae17-9b8665fb4110 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.234208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:27.247469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4709327-c891-4694-960e-824ed64bc457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.247993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4709327-c891-4694-960e-824ed64bc457 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.248266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4709327-c891-4694-960e-824ed64bc457 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.248534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4709327-c891-4694-960e-824ed64bc457 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.249198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4709327-c891-4694-960e-824ed64bc457 None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:27.249498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4709327-c891-4694-960e-824ed64bc457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.250849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4709327-c891-4694-960e-824ed64bc457 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:27.253816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:27.264680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 387/774] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:27.264680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc230-c760-43be-947f-945e515b7c17 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.264680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc230-c760-43be-947f-945e515b7c17 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.264680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc230-c760-43be-947f-945e515b7c17 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.264680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 388/775] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:27.264680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dc230-c760-43be-947f-945e515b7c17 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.292579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72507692-fa7a-4fdf-9d97-bd0fb17ce96d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.329667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6613-1872-4f46-9654-baa37fcb853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.330661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6613-1872-4f46-9654-baa37fcb853c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.331265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6613-1872-4f46-9654-baa37fcb853c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.331548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6613-1872-4f46-9654-baa37fcb853c None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.332298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd6613-1872-4f46-9654-baa37fcb853c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:27.332605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd6613-1872-4f46-9654-baa37fcb853c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.334035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd6613-1872-4f46-9654-baa37fcb853c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:27.352782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 388/776] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:27.353553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6613-1872-4f46-9654-baa37fcb853c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.371865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dc230-c760-43be-947f-945e515b7c17 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.408718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb4052-f5d2-4ae8-9801-0c8b976d2cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.409326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb4052-f5d2-4ae8-9801-0c8b976d2cba None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.409604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb4052-f5d2-4ae8-9801-0c8b976d2cba None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.409872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb4052-f5d2-4ae8-9801-0c8b976d2cba None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.410472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb4052-f5d2-4ae8-9801-0c8b976d2cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.411869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb4052-f5d2-4ae8-9801-0c8b976d2cba None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:27.417427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 389/777] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 10 09:51:27 2022] GET /identity/v3/services => generated 772 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:27.418719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0a7b8-dfbd-4b77-aa37-27f9735a3e56 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.419020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0a7b8-dfbd-4b77-aa37-27f9735a3e56 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.419294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0a7b8-dfbd-4b77-aa37-27f9735a3e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.439357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-40d0a7b8-dfbd-4b77-aa37-27f9735a3e56 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:27.440591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-40d0a7b8-dfbd-4b77-aa37-27f9735a3e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:27.458569 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6613-1872-4f46-9654-baa37fcb853c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.490932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:27.493022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:27.500177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76657932-0549-42c5-845b-4095dcffc3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.500752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76657932-0549-42c5-845b-4095dcffc3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.501035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76657932-0549-42c5-845b-4095dcffc3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.501313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76657932-0549-42c5-845b-4095dcffc3c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.512835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76657932-0549-42c5-845b-4095dcffc3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.514322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76657932-0549-42c5-845b-4095dcffc3c6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:27.521083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 390/778] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:27.522875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0a7b8-dfbd-4b77-aa37-27f9735a3e56 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-76657932-0549-42c5-845b-4095dcffc3c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:27.530344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:27.531588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:27.532471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:27.532471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:27.532471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:27.532471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 389/779] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:27.533061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56587359-25d6-4341-8092-205075f30629 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.533334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56587359-25d6-4341-8092-205075f30629 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.533610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56587359-25d6-4341-8092-205075f30629 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.539219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 390/780] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:27.540174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56587359-25d6-4341-8092-205075f30629 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.645222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0a7b8-dfbd-4b77-aa37-27f9735a3e56 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.652417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56587359-25d6-4341-8092-205075f30629 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.681696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f744caf-9947-4548-80fa-2dfcacc0029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.682349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f744caf-9947-4548-80fa-2dfcacc0029e None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.682630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f744caf-9947-4548-80fa-2dfcacc0029e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.682904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f744caf-9947-4548-80fa-2dfcacc0029e None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/904f7f036565419b9fdecc5e47acb45c/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.684834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f744caf-9947-4548-80fa-2dfcacc0029e None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=904f7f036565419b9fdecc5e47acb45c, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.686340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f744caf-9947-4548-80fa-2dfcacc0029e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:27.687210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca50e6b-a7a0-4c81-af7d-8c4df40aabc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.687814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca50e6b-a7a0-4c81-af7d-8c4df40aabc3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.688096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca50e6b-a7a0-4c81-af7d-8c4df40aabc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.688372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca50e6b-a7a0-4c81-af7d-8c4df40aabc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.689006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca50e6b-a7a0-4c81-af7d-8c4df40aabc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:27.689312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca50e6b-a7a0-4c81-af7d-8c4df40aabc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.690788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca50e6b-a7a0-4c81-af7d-8c4df40aabc3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:27.700806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 391/781] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:51:27 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/904f7f036565419b9fdecc5e47acb45c/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:51:27.702188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44136325-e5e3-4a02-a4c3-5d874913c423 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.702544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44136325-e5e3-4a02-a4c3-5d874913c423 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.702892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44136325-e5e3-4a02-a4c3-5d874913c423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.720261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 391/782] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:27.721573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca50e6b-a7a0-4c81-af7d-8c4df40aabc3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.726904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-44136325-e5e3-4a02-a4c3-5d874913c423 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:27.728288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-44136325-e5e3-4a02-a4c3-5d874913c423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:27.766106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:27.787479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:27.812673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 392/783] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:27.814297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44136325-e5e3-4a02-a4c3-5d874913c423 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.858858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca50e6b-a7a0-4c81-af7d-8c4df40aabc3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.901810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247c784-ea50-4a87-8f93-14426e8bb447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.902912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247c784-ea50-4a87-8f93-14426e8bb447 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.904198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247c784-ea50-4a87-8f93-14426e8bb447 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.904567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247c784-ea50-4a87-8f93-14426e8bb447 None admin] PATH_INFO: `/v3/services/network` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.905297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247c784-ea50-4a87-8f93-14426e8bb447 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.906801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247c784-ea50-4a87-8f93-14426e8bb447 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-1247c784-ea50-4a87-8f93-14426e8bb447 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:27.916540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:27.917540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:27.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 392/784] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:27 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:27.922186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247c784-ea50-4a87-8f93-14426e8bb447 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:27.937763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44136325-e5e3-4a02-a4c3-5d874913c423 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:27.976906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0851a-bb32-4e47-a8d1-7803579d79be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:27.977869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0851a-bb32-4e47-a8d1-7803579d79be None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:27.978429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0851a-bb32-4e47-a8d1-7803579d79be None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:27.978881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0851a-bb32-4e47-a8d1-7803579d79be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:27.987000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0851a-bb32-4e47-a8d1-7803579d79be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:27.988773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0851a-bb32-4e47-a8d1-7803579d79be None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-46e0851a-bb32-4e47-a8d1-7803579d79be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:28.001749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:28.002730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:28.003721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:28.006498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 393/785] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:51:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:28.007527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0851a-bb32-4e47-a8d1-7803579d79be None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.029591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247c784-ea50-4a87-8f93-14426e8bb447 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.039840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:28.043149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:28.069216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33dc6-2b9d-4d5b-b891-136a228e1110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.070035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33dc6-2b9d-4d5b-b891-136a228e1110 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.070374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33dc6-2b9d-4d5b-b891-136a228e1110 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.070709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33dc6-2b9d-4d5b-b891-136a228e1110 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.082319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee33dc6-2b9d-4d5b-b891-136a228e1110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-dee33dc6-2b9d-4d5b-b891-136a228e1110 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:28.088041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:28.088988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:28.090010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 393/786] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:28.090884 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33dc6-2b9d-4d5b-b891-136a228e1110 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.134004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0851a-bb32-4e47-a8d1-7803579d79be None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.170082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bbf67-6aad-4fbe-9265-2363720822ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.170709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bbf67-6aad-4fbe-9265-2363720822ed None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.171006 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bbf67-6aad-4fbe-9265-2363720822ed None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.171323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bbf67-6aad-4fbe-9265-2363720822ed None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.171948 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bbf67-6aad-4fbe-9265-2363720822ed None admin] RBAC: Adding query filter params (name=network) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:28.172247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bbf67-6aad-4fbe-9265-2363720822ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:28.173662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bbf67-6aad-4fbe-9265-2363720822ed None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.178580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 394/787] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:28.179643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bbf67-6aad-4fbe-9265-2363720822ed None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.233888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33dc6-2b9d-4d5b-b891-136a228e1110 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.279171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e221e2-a7f2-4ded-b2dd-04d0ea3ef6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.280363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e221e2-a7f2-4ded-b2dd-04d0ea3ef6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.281164 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e221e2-a7f2-4ded-b2dd-04d0ea3ef6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.281929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e221e2-a7f2-4ded-b2dd-04d0ea3ef6f5 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.283051 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e221e2-a7f2-4ded-b2dd-04d0ea3ef6f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:28.283872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e221e2-a7f2-4ded-b2dd-04d0ea3ef6f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:28.285817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e221e2-a7f2-4ded-b2dd-04d0ea3ef6f5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.286235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bbf67-6aad-4fbe-9265-2363720822ed None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.289392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:28.308823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 394/788] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:28.310031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e221e2-a7f2-4ded-b2dd-04d0ea3ef6f5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.332197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808e695-d69f-4581-8926-549317965b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.333262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808e695-d69f-4581-8926-549317965b26 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.333541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808e695-d69f-4581-8926-549317965b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.333815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808e695-d69f-4581-8926-549317965b26 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.334683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808e695-d69f-4581-8926-549317965b26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:28.335106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808e695-d69f-4581-8926-549317965b26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:28.337335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808e695-d69f-4581-8926-549317965b26 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.347957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:28.353937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 395/789] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:28.353937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808e695-d69f-4581-8926-549317965b26 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.456928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e221e2-a7f2-4ded-b2dd-04d0ea3ef6f5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.484141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808e695-d69f-4581-8926-549317965b26 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.511410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe842aa-10fd-42c1-a9fc-d59b4ecf7174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.511410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe842aa-10fd-42c1-a9fc-d59b4ecf7174 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.511410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe842aa-10fd-42c1-a9fc-d59b4ecf7174 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.511410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe842aa-10fd-42c1-a9fc-d59b4ecf7174 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.511804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe842aa-10fd-42c1-a9fc-d59b4ecf7174 None admin] RBAC: Adding query filter params (type=network) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:28.511804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe842aa-10fd-42c1-a9fc-d59b4ecf7174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:28.512746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe842aa-10fd-42c1-a9fc-d59b4ecf7174 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.518265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 395/790] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:28.519201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe842aa-10fd-42c1-a9fc-d59b4ecf7174 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.527595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7321b51-f441-471e-84a1-8565ccbd0ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.528452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7321b51-f441-471e-84a1-8565ccbd0ace None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.528870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7321b51-f441-471e-84a1-8565ccbd0ace None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.529279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7321b51-f441-471e-84a1-8565ccbd0ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.540857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7321b51-f441-471e-84a1-8565ccbd0ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:28.542491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7321b51-f441-471e-84a1-8565ccbd0ace None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-f7321b51-f441-471e-84a1-8565ccbd0ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:28.558408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:28.559589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:28.560461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:28.560461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:28.560461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:28.566868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 396/791] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:28.568029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7321b51-f441-471e-84a1-8565ccbd0ace None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.578414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:28.623482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:28.633028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe842aa-10fd-42c1-a9fc-d59b4ecf7174 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.667198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b68c1-89ad-443e-a891-a936a538df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.667871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b68c1-89ad-443e-a891-a936a538df7d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.668157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b68c1-89ad-443e-a891-a936a538df7d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.668428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b68c1-89ad-443e-a891-a936a538df7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.675800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7321b51-f441-471e-84a1-8565ccbd0ace None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.679839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b68c1-89ad-443e-a891-a936a538df7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:28.680908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b68c1-89ad-443e-a891-a936a538df7d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-d76b68c1-89ad-443e-a891-a936a538df7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:28.696239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:28.697172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:28.698057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:28.698057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:28.698057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:28.702031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 396/792] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:28.702878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b68c1-89ad-443e-a891-a936a538df7d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.715783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc515c-dd3f-40e0-af1c-e2ce065c815d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.716725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc515c-dd3f-40e0-af1c-e2ce065c815d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.717196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc515c-dd3f-40e0-af1c-e2ce065c815d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.717705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc515c-dd3f-40e0-af1c-e2ce065c815d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.718586 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc515c-dd3f-40e0-af1c-e2ce065c815d None admin] RBAC: Adding query filter params (interface=public, service_id=72aeaf539483430e87b9da11785813a1, region_id=RegionOne) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:28.719181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc515c-dd3f-40e0-af1c-e2ce065c815d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:28.720723 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc515c-dd3f-40e0-af1c-e2ce065c815d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.728821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 397/793] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/endpoints?service_id=72aeaf539483430e87b9da11785813a1&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:28.729794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc515c-dd3f-40e0-af1c-e2ce065c815d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.802070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:28.805873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b68c1-89ad-443e-a891-a936a538df7d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.836279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc515c-dd3f-40e0-af1c-e2ce065c815d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:28.840132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f4eab-9345-44fd-869f-78009affd264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.840742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f4eab-9345-44fd-869f-78009affd264 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.841024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f4eab-9345-44fd-869f-78009affd264 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.841300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f4eab-9345-44fd-869f-78009affd264 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.841941 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f4eab-9345-44fd-869f-78009affd264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:28.842245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f4eab-9345-44fd-869f-78009affd264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:28.843694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f4eab-9345-44fd-869f-78009affd264 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.860498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 397/794] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:28.860498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f4eab-9345-44fd-869f-78009affd264 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.871300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:28.880421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0226471-4530-41ba-ac88-48b1eb8a7cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:28.881327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0226471-4530-41ba-ac88-48b1eb8a7cac None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:28.881819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0226471-4530-41ba-ac88-48b1eb8a7cac None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:28.882842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0226471-4530-41ba-ac88-48b1eb8a7cac None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:28.883931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0226471-4530-41ba-ac88-48b1eb8a7cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:28.884424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0226471-4530-41ba-ac88-48b1eb8a7cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:28.886087 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0226471-4530-41ba-ac88-48b1eb8a7cac None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:28.904753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 398/795] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:28.905650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0226471-4530-41ba-ac88-48b1eb8a7cac None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:28.983481 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f4eab-9345-44fd-869f-78009affd264 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:29.012644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0226471-4530-41ba-ac88-48b1eb8a7cac None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:29.022920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255de4c-ac85-44e0-8ce2-567cac7bf08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:29.023663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255de4c-ac85-44e0-8ce2-567cac7bf08c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.023950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255de4c-ac85-44e0-8ce2-567cac7bf08c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.024215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255de4c-ac85-44e0-8ce2-567cac7bf08c None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.024822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255de4c-ac85-44e0-8ce2-567cac7bf08c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:29.026333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255de4c-ac85-44e0-8ce2-567cac7bf08c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:29.032359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 398/796] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:29.033375 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255de4c-ac85-44e0-8ce2-567cac7bf08c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:29.047492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:29.065267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87886b-b83d-4e2a-83fa-ac90ef0f401c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:29.065838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87886b-b83d-4e2a-83fa-ac90ef0f401c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.066121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87886b-b83d-4e2a-83fa-ac90ef0f401c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.066396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87886b-b83d-4e2a-83fa-ac90ef0f401c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.082182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87886b-b83d-4e2a-83fa-ac90ef0f401c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:29.083737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87886b-b83d-4e2a-83fa-ac90ef0f401c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-9c87886b-b83d-4e2a-83fa-ac90ef0f401c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:29.109828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:29.110767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:29.111676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:29.111676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:29.111676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:29.111676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:29.111676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:29.111676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:29.111676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:29.111676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:29.111676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 399/797] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:29.119346 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87886b-b83d-4e2a-83fa-ac90ef0f401c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:29.124934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:29.168607 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255de4c-ac85-44e0-8ce2-567cac7bf08c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:29.206499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c09c3-4e51-4937-b78b-e7e70a6e1f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:29.207157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c09c3-4e51-4937-b78b-e7e70a6e1f84 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.207437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c09c3-4e51-4937-b78b-e7e70a6e1f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.207709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c09c3-4e51-4937-b78b-e7e70a6e1f84 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.215414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c09c3-4e51-4937-b78b-e7e70a6e1f84 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:29.217466 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c09c3-4e51-4937-b78b-e7e70a6e1f84 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e99c09c3-4e51-4937-b78b-e7e70a6e1f84 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:29.229111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:29.231184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:29.233354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:29.235177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 399/798] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:29.238550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c09c3-4e51-4937-b78b-e7e70a6e1f84 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:29.261323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87886b-b83d-4e2a-83fa-ac90ef0f401c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:29.310632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555d92b-cb8a-4688-82f6-8450905238f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:29.311286 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555d92b-cb8a-4688-82f6-8450905238f5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.311566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555d92b-cb8a-4688-82f6-8450905238f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.311838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555d92b-cb8a-4688-82f6-8450905238f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.312539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555d92b-cb8a-4688-82f6-8450905238f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:29.312906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555d92b-cb8a-4688-82f6-8450905238f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:29.314445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555d92b-cb8a-4688-82f6-8450905238f5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:29.344154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 400/799] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:29.346463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c09c3-4e51-4937-b78b-e7e70a6e1f84 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:29.350008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555d92b-cb8a-4688-82f6-8450905238f5 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:29.375471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:29.383350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8685b7-6639-4a97-afe6-f5cc10a9041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:29.384192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8685b7-6639-4a97-afe6-f5cc10a9041d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.384673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8685b7-6639-4a97-afe6-f5cc10a9041d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.385398 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8685b7-6639-4a97-afe6-f5cc10a9041d None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.386317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8685b7-6639-4a97-afe6-f5cc10a9041d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:29.386801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8685b7-6639-4a97-afe6-f5cc10a9041d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:29.388450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8685b7-6639-4a97-afe6-f5cc10a9041d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:29.414974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 400/800] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 10 09:51:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:29.414974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8685b7-6639-4a97-afe6-f5cc10a9041d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:29.443415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:29.542560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555d92b-cb8a-4688-82f6-8450905238f5 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:29.543659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8685b7-6639-4a97-afe6-f5cc10a9041d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:29.579236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25692a-9bbc-4a03-a042-a91b7a25e9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:29.580367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25692a-9bbc-4a03-a042-a91b7a25e9b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.580910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25692a-9bbc-4a03-a042-a91b7a25e9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.581375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25692a-9bbc-4a03-a042-a91b7a25e9b4 None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/36f9577e047a406480fadc4b9f8c6234/roles/bacea3b4b26e48a598b75ea1b301fafb` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.587209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a552f-0313-4f66-8872-1d00f8609ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:29.587846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a552f-0313-4f66-8872-1d00f8609ecb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.588125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a552f-0313-4f66-8872-1d00f8609ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.588394 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a552f-0313-4f66-8872-1d00f8609ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.598294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25692a-9bbc-4a03-a042-a91b7a25e9b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=36f9577e047a406480fadc4b9f8c6234, role_id=bacea3b4b26e48a598b75ea1b301fafb)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:29.599998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25692a-9bbc-4a03-a042-a91b7a25e9b4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:29.600107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a552f-0313-4f66-8872-1d00f8609ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:29.602174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a552f-0313-4f66-8872-1d00f8609ecb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:29.614524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 401/801] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:51:29 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/36f9577e047a406480fadc4b9f8c6234/roles/bacea3b4b26e48a598b75ea1b301fafb => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:51:29.616071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6970779-7011-4834-8316-84ad96240316 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.616580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6970779-7011-4834-8316-84ad96240316 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.617117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6970779-7011-4834-8316-84ad96240316 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.618843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 402/802] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:29.624506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb6c07-2f92-4ed2-b749-191d2f53f7a0 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.625078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb6c07-2f92-4ed2-b749-191d2f53f7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.625549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb6c07-2f92-4ed2-b749-191d2f53f7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-cb9a552f-0313-4f66-8872-1d00f8609ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:29.633743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:29.635119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:29.635996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:29.635996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:29.635996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:29.635996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 401/803] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:29.648585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a552f-0313-4f66-8872-1d00f8609ecb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:29.649857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-5ceb6c07-2f92-4ed2-b749-191d2f53f7a0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:29.651360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5ceb6c07-2f92-4ed2-b749-191d2f53f7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:29.726192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:29.796883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 403/804] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:29.828197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb6c07-2f92-4ed2-b749-191d2f53f7a0 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:29.843303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a552f-0313-4f66-8872-1d00f8609ecb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:29.865770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:29.889599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaf793-2390-4369-8df2-15fb55a17931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:29.890484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaf793-2390-4369-8df2-15fb55a17931 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:29.890768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaf793-2390-4369-8df2-15fb55a17931 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:29.891066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaf793-2390-4369-8df2-15fb55a17931 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:29.891715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaf793-2390-4369-8df2-15fb55a17931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:29.892018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaf793-2390-4369-8df2-15fb55a17931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:29.893843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaf793-2390-4369-8df2-15fb55a17931 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:29.914717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 402/805] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:29.916001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaf793-2390-4369-8df2-15fb55a17931 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:29.974203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb6c07-2f92-4ed2-b749-191d2f53f7a0 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:30.009230 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b65605-e980-4268-9cf9-2a653964c3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:30.009858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b65605-e980-4268-9cf9-2a653964c3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.010137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b65605-e980-4268-9cf9-2a653964c3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.010405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b65605-e980-4268-9cf9-2a653964c3d3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.011009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b65605-e980-4268-9cf9-2a653964c3d3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:30.012553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b65605-e980-4268-9cf9-2a653964c3d3 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-28b65605-e980-4268-9cf9-2a653964c3d3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:30.021469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:30.022455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:30.023867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 404/806] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Aug 10 09:51:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:30.024667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eb9f6-8f80-43ff-abfc-347ad72933b1 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.025246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eb9f6-8f80-43ff-abfc-347ad72933b1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.025246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eb9f6-8f80-43ff-abfc-347ad72933b1 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.026653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 405/807] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:30.027396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:30.031997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eb9f6-8f80-43ff-abfc-347ad72933b1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:30.037345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaf793-2390-4369-8df2-15fb55a17931 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:30.079347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded3382-559b-4161-ba6e-3d5727f87bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:30.080226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded3382-559b-4161-ba6e-3d5727f87bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.080690 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded3382-559b-4161-ba6e-3d5727f87bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.081444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded3382-559b-4161-ba6e-3d5727f87bf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.090278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded3382-559b-4161-ba6e-3d5727f87bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:30.091921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded3382-559b-4161-ba6e-3d5727f87bf0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6ded3382-559b-4161-ba6e-3d5727f87bf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:30.105736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:30.106896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:30.107959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:30.107959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:30.107959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:30.107959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:30.107959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:30.107959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:30.107959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:30.107959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:30.108829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 403/808] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:30.110116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81397cd-72bf-4162-b0ca-c1656a795dfb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.110461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81397cd-72bf-4162-b0ca-c1656a795dfb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.110803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81397cd-72bf-4162-b0ca-c1656a795dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.133270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d81397cd-72bf-4162-b0ca-c1656a795dfb None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:30.135158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d81397cd-72bf-4162-b0ca-c1656a795dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:30.151472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:30.159822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eb9f6-8f80-43ff-abfc-347ad72933b1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:30.209848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18dcaf7-6271-42a5-8241-f494ad9fa248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:30.210919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dcaf7-6271-42a5-8241-f494ad9fa248 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.211523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dcaf7-6271-42a5-8241-f494ad9fa248 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.212045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dcaf7-6271-42a5-8241-f494ad9fa248 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.212874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18dcaf7-6271-42a5-8241-f494ad9fa248 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:30.213408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18dcaf7-6271-42a5-8241-f494ad9fa248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:30.215087 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18dcaf7-6271-42a5-8241-f494ad9fa248 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:30.219284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 404/809] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:30.220254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81397cd-72bf-4162-b0ca-c1656a795dfb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:30.220959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 406/810] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 10 09:51:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:30.226946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18dcaf7-6271-42a5-8241-f494ad9fa248 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:30.326053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:30.347321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81397cd-72bf-4162-b0ca-c1656a795dfb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:30.349945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18dcaf7-6271-42a5-8241-f494ad9fa248 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:30.388768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d7de1-91a5-423b-bb87-1102f2675315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:30.389813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d7de1-91a5-423b-bb87-1102f2675315 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.390114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d7de1-91a5-423b-bb87-1102f2675315 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.390383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d7de1-91a5-423b-bb87-1102f2675315 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.391296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d7de1-91a5-423b-bb87-1102f2675315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:30.391602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d7de1-91a5-423b-bb87-1102f2675315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:30.393695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d7de1-91a5-423b-bb87-1102f2675315 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:30.397275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0cd0e-4268-41e5-aae8-04a8dcedbf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:30.398030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0cd0e-4268-41e5-aae8-04a8dcedbf99 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.398373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0cd0e-4268-41e5-aae8-04a8dcedbf99 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.398713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0cd0e-4268-41e5-aae8-04a8dcedbf99 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.399455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0cd0e-4268-41e5-aae8-04a8dcedbf99 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:30.399824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0cd0e-4268-41e5-aae8-04a8dcedbf99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:30.401836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0cd0e-4268-41e5-aae8-04a8dcedbf99 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:30.407194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 405/811] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:30.408060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d7de1-91a5-423b-bb87-1102f2675315 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:30.419594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 407/812] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 10 09:51:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:30.419594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0cd0e-4268-41e5-aae8-04a8dcedbf99 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:30.447076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:30.535454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d7de1-91a5-423b-bb87-1102f2675315 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:30.549917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0cd0e-4268-41e5-aae8-04a8dcedbf99 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:30.577928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf23494-fff1-42ae-9326-6f164aba8fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:30.578555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23494-fff1-42ae-9326-6f164aba8fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.578831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23494-fff1-42ae-9326-6f164aba8fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.579131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23494-fff1-42ae-9326-6f164aba8fa4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.591871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf23494-fff1-42ae-9326-6f164aba8fa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:30.593044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888b97-05e0-43fd-83ec-8b479f430105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:30.593875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888b97-05e0-43fd-83ec-8b479f430105 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.594308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888b97-05e0-43fd-83ec-8b479f430105 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.594724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888b97-05e0-43fd-83ec-8b479f430105 None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/af29f3451f654f53ac98239a2b28d128/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.596827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888b97-05e0-43fd-83ec-8b479f430105 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=af29f3451f654f53ac98239a2b28d128, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-4cf23494-fff1-42ae-9326-6f164aba8fa4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:30.596945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:30.597994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:30.599028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:30.601793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888b97-05e0-43fd-83ec-8b479f430105 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:30.603310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 406/813] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:30.604116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf23494-fff1-42ae-9326-6f164aba8fa4 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:30.615920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:30.628326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 408/814] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:51:30 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/af29f3451f654f53ac98239a2b28d128/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:51:30.628326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888b97-05e0-43fd-83ec-8b479f430105 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:30.791883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf23494-fff1-42ae-9326-6f164aba8fa4 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:30.796793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:30.834181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888b97-05e0-43fd-83ec-8b479f430105 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:30.856717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b679e-bd99-4806-b6d7-dadd5cc4a905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:30.857436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b679e-bd99-4806-b6d7-dadd5cc4a905 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.857781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b679e-bd99-4806-b6d7-dadd5cc4a905 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.858116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b679e-bd99-4806-b6d7-dadd5cc4a905 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.858878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b679e-bd99-4806-b6d7-dadd5cc4a905 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:30.860356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b679e-bd99-4806-b6d7-dadd5cc4a905 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:30.889263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdc118-1518-484e-a8fc-2adc721c78b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:30.890827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdc118-1518-484e-a8fc-2adc721c78b4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:30.891701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdc118-1518-484e-a8fc-2adc721c78b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:30.892616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdc118-1518-484e-a8fc-2adc721c78b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:30.893796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdc118-1518-484e-a8fc-2adc721c78b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:30.894334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdc118-1518-484e-a8fc-2adc721c78b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:30.894895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 407/815] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:51:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:30.896391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdc118-1518-484e-a8fc-2adc721c78b4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:30.911784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 409/816] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:30.918722 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b679e-bd99-4806-b6d7-dadd5cc4a905 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:31.007713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:31.068002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b679e-bd99-4806-b6d7-dadd5cc4a905 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:31.106934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ce679-14dd-499d-b695-2813046c4e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:31.107925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ce679-14dd-499d-b695-2813046c4e01 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:31.108443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ce679-14dd-499d-b695-2813046c4e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:31.108890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ce679-14dd-499d-b695-2813046c4e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:31.120011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ce679-14dd-499d-b695-2813046c4e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:31.121833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ce679-14dd-499d-b695-2813046c4e01 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c51ce679-14dd-499d-b695-2813046c4e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:31.137514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:31.138507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:31.139719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:31.139719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:31.139719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:31.141383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 408/817] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:31.147180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdc118-1518-484e-a8fc-2adc721c78b4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:31.255399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdc118-1518-484e-a8fc-2adc721c78b4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:31.294232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc8727-d07e-40c3-8185-800dbf934575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:31.294850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc8727-d07e-40c3-8185-800dbf934575 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:31.295155 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc8727-d07e-40c3-8185-800dbf934575 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:31.295423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc8727-d07e-40c3-8185-800dbf934575 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:31.296064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc8727-d07e-40c3-8185-800dbf934575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:31.296361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc8727-d07e-40c3-8185-800dbf934575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:31.297719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc8727-d07e-40c3-8185-800dbf934575 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:31.309947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 410/818] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:31.315677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ce679-14dd-499d-b695-2813046c4e01 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:31.364054 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a39dd3-6655-4cbb-b201-87fa6f27ac1d None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:31.364370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a39dd3-6655-4cbb-b201-87fa6f27ac1d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:31.364552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a39dd3-6655-4cbb-b201-87fa6f27ac1d None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:31.366145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 411/819] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:31.371307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e4a60-d375-494b-a446-e9f2f311a870 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:31.371615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e4a60-d375-494b-a446-e9f2f311a870 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:31.371862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e4a60-d375-494b-a446-e9f2f311a870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:31.391324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:31.395671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-577e4a60-d375-494b-a446-e9f2f311a870 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:31.396920 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-577e4a60-d375-494b-a446-e9f2f311a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:31.490054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ce679-14dd-499d-b695-2813046c4e01 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:31.501485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:31.529214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30d531-c3d2-46b6-8e3e-c86e9b56bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:31.532767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30d531-c3d2-46b6-8e3e-c86e9b56bd7f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:31.532767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30d531-c3d2-46b6-8e3e-c86e9b56bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:31.532767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30d531-c3d2-46b6-8e3e-c86e9b56bd7f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:31.538692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30d531-c3d2-46b6-8e3e-c86e9b56bd7f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:31.540281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30d531-c3d2-46b6-8e3e-c86e9b56bd7f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-4f30d531-c3d2-46b6-8e3e-c86e9b56bd7f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:31.552093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:31.553040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:31.554503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 409/820] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 10 09:51:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:31.561364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30d531-c3d2-46b6-8e3e-c86e9b56bd7f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:31.583015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 412/821] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:31.618160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e4a60-d375-494b-a446-e9f2f311a870 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:31.740265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30d531-c3d2-46b6-8e3e-c86e9b56bd7f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:31.806651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e4a60-d375-494b-a446-e9f2f311a870 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:31.808615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae75c3f-dfce-4249-b560-87ae8e684864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:31.809162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae75c3f-dfce-4249-b560-87ae8e684864 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:31.809426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae75c3f-dfce-4249-b560-87ae8e684864 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:31.809697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae75c3f-dfce-4249-b560-87ae8e684864 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:31.810426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae75c3f-dfce-4249-b560-87ae8e684864 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:31.811460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae75c3f-dfce-4249-b560-87ae8e684864 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:31.812158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae75c3f-dfce-4249-b560-87ae8e684864 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:31.844726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 410/822] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 10 09:51:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:31.852724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a75599-8f9a-4e87-b795-bd3a10ead6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:31.854249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae75c3f-dfce-4249-b560-87ae8e684864 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:31.855154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a75599-8f9a-4e87-b795-bd3a10ead6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:31.855630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a75599-8f9a-4e87-b795-bd3a10ead6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:31.856416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a75599-8f9a-4e87-b795-bd3a10ead6d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:31.857768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a75599-8f9a-4e87-b795-bd3a10ead6d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:31.861173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a75599-8f9a-4e87-b795-bd3a10ead6d5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:31.865607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-b2a75599-8f9a-4e87-b795-bd3a10ead6d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:31.880998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:31.882079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 09:51:31.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:31.886645 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 413/823] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:31 2022] GET /identity/v3/services/network => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:31.892472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a75599-8f9a-4e87-b795-bd3a10ead6d5 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:31.916300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:32.027115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae75c3f-dfce-4249-b560-87ae8e684864 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:32.030975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a75599-8f9a-4e87-b795-bd3a10ead6d5 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:32.065189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174c710-3a6d-4caa-a463-d36a1905415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:32.065816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174c710-3a6d-4caa-a463-d36a1905415c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.066098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174c710-3a6d-4caa-a463-d36a1905415c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.066559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174c710-3a6d-4caa-a463-d36a1905415c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.069303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad2f47-ccaa-4640-8f47-2228f2dd65de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:32.069979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad2f47-ccaa-4640-8f47-2228f2dd65de None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.070257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad2f47-ccaa-4640-8f47-2228f2dd65de None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.070532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad2f47-ccaa-4640-8f47-2228f2dd65de None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.071232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad2f47-ccaa-4640-8f47-2228f2dd65de None admin] RBAC: Adding query filter params (name=network) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:32.071535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad2f47-ccaa-4640-8f47-2228f2dd65de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:32.072938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad2f47-ccaa-4640-8f47-2228f2dd65de None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:32.078600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 414/824] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:32.080007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2885bec-d488-4f07-8d6f-129a588145f8 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.080288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2885bec-d488-4f07-8d6f-129a588145f8 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.080557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2885bec-d488-4f07-8d6f-129a588145f8 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.085865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 415/825] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:32.086652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2885bec-d488-4f07-8d6f-129a588145f8 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:32.141909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174c710-3a6d-4caa-a463-d36a1905415c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:32.143401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174c710-3a6d-4caa-a463-d36a1905415c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:32.151860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:32.172948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:32.195031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2885bec-d488-4f07-8d6f-129a588145f8 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:32.230316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0da53f-b58e-40a4-baff-bee8b11feaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:32.231233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0da53f-b58e-40a4-baff-bee8b11feaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.231506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0da53f-b58e-40a4-baff-bee8b11feaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.231776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0da53f-b58e-40a4-baff-bee8b11feaf9 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.232406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0da53f-b58e-40a4-baff-bee8b11feaf9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:32.232702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0da53f-b58e-40a4-baff-bee8b11feaf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:32.234050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0da53f-b58e-40a4-baff-bee8b11feaf9 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:32.242995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 416/826] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:32.242995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b3d86-abbb-4079-9f91-e6789f2d206f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.242995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b3d86-abbb-4079-9f91-e6789f2d206f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.242995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b3d86-abbb-4079-9f91-e6789f2d206f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.260316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f69b3d86-abbb-4079-9f91-e6789f2d206f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:32.261489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f69b3d86-abbb-4079-9f91-e6789f2d206f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:32.262589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 411/827] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:51:32.263466 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174c710-3a6d-4caa-a463-d36a1905415c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:32.345013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 417/828] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:32.346199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b3d86-abbb-4079-9f91-e6789f2d206f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:32.386981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174c710-3a6d-4caa-a463-d36a1905415c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:32.416601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:32.425827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b58ea-9b03-4a73-adb5-8df5fb7fa148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:32.426445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b58ea-9b03-4a73-adb5-8df5fb7fa148 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.426723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b58ea-9b03-4a73-adb5-8df5fb7fa148 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.427032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b58ea-9b03-4a73-adb5-8df5fb7fa148 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.427699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b58ea-9b03-4a73-adb5-8df5fb7fa148 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:32.429086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b58ea-9b03-4a73-adb5-8df5fb7fa148 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:32.435786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:32.459012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 412/829] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:51:32 2022] POST /identity/v3/endpoints => generated 337 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:32.459012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b58ea-9b03-4a73-adb5-8df5fb7fa148 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:32.517833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b3d86-abbb-4079-9f91-e6789f2d206f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:32.576312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28736a68-6e20-4240-a062-ce1e2bea450e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:32.578609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28736a68-6e20-4240-a062-ce1e2bea450e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.579068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28736a68-6e20-4240-a062-ce1e2bea450e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.582967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28736a68-6e20-4240-a062-ce1e2bea450e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.597792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28736a68-6e20-4240-a062-ce1e2bea450e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:32.599755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28736a68-6e20-4240-a062-ce1e2bea450e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-28736a68-6e20-4240-a062-ce1e2bea450e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:32.617783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:32.619256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:32.620260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:32.620260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:32.620260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:32.627205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 418/830] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:32.628923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28736a68-6e20-4240-a062-ce1e2bea450e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:32.688314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b58ea-9b03-4a73-adb5-8df5fb7fa148 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:32.734156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ed30a-fcf1-43fb-8bd4-45879308b016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:32.735420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ed30a-fcf1-43fb-8bd4-45879308b016 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.735847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ed30a-fcf1-43fb-8bd4-45879308b016 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.736572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ed30a-fcf1-43fb-8bd4-45879308b016 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.749776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ed30a-fcf1-43fb-8bd4-45879308b016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-405ed30a-fcf1-43fb-8bd4-45879308b016 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:32.758681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:32.759859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:51:32.760837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:32.761521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28736a68-6e20-4240-a062-ce1e2bea450e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:32.762766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:32.765033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 413/831] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:51:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:32.771347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ed30a-fcf1-43fb-8bd4-45879308b016 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:32.806570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bd773-ce31-4468-9cb3-57b3a0d4f431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:32.807647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bd773-ce31-4468-9cb3-57b3a0d4f431 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.808391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bd773-ce31-4468-9cb3-57b3a0d4f431 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.808936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bd773-ce31-4468-9cb3-57b3a0d4f431 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.809718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bd773-ce31-4468-9cb3-57b3a0d4f431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:32.810301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bd773-ce31-4468-9cb3-57b3a0d4f431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:32.812293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bd773-ce31-4468-9cb3-57b3a0d4f431 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:32.827816 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 419/832] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:32.833203 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:32.836583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bd773-ce31-4468-9cb3-57b3a0d4f431 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:32.907596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ed30a-fcf1-43fb-8bd4-45879308b016 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:32.946291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92d8d0-a10d-4f38-9c7e-37cf33a4294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:32.947245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92d8d0-a10d-4f38-9c7e-37cf33a4294d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:32.947677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92d8d0-a10d-4f38-9c7e-37cf33a4294d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:32.948085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92d8d0-a10d-4f38-9c7e-37cf33a4294d None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:32.948891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92d8d0-a10d-4f38-9c7e-37cf33a4294d None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:32.949329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92d8d0-a10d-4f38-9c7e-37cf33a4294d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:32.950835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92d8d0-a10d-4f38-9c7e-37cf33a4294d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:32.960229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 414/833] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:51:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:32.970319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92d8d0-a10d-4f38-9c7e-37cf33a4294d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:32.980086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bd773-ce31-4468-9cb3-57b3a0d4f431 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.023854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350afe8-59cc-4a4c-a329-acbefcedeb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.024890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350afe8-59cc-4a4c-a329-acbefcedeb8c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.025365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350afe8-59cc-4a4c-a329-acbefcedeb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.025829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350afe8-59cc-4a4c-a329-acbefcedeb8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.033333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350afe8-59cc-4a4c-a329-acbefcedeb8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:33.035219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350afe8-59cc-4a4c-a329-acbefcedeb8c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-6350afe8-59cc-4a4c-a329-acbefcedeb8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:33.047782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:33.048689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:33.049596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:33.049596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:33.049596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:33.049596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:33.049596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:33.049596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:33.049596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:33.049596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:33.049596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 420/834] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:33.050176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21acb0-5640-4288-8e56-9673e2a6d6f9 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.050381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21acb0-5640-4288-8e56-9673e2a6d6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.050650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21acb0-5640-4288-8e56-9673e2a6d6f9 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.052011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 421/835] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:33.055956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21acb0-5640-4288-8e56-9673e2a6d6f9 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.082218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:33.103055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92d8d0-a10d-4f38-9c7e-37cf33a4294d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.110856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:33.140222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b92940-6f2d-46d0-ba0a-f6dcea5122cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.140965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b92940-6f2d-46d0-ba0a-f6dcea5122cf None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.141125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b92940-6f2d-46d0-ba0a-f6dcea5122cf None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.141381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b92940-6f2d-46d0-ba0a-f6dcea5122cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.152626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b92940-6f2d-46d0-ba0a-f6dcea5122cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:33.153964 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b92940-6f2d-46d0-ba0a-f6dcea5122cf None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:33.164979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21acb0-5640-4288-8e56-9673e2a6d6f9 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-11b92940-6f2d-46d0-ba0a-f6dcea5122cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:33.169037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:33.169975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:33.175729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:33.175729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:33.175729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:33.175729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 415/836] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:33.175729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334142eb-e56b-4971-b89c-cedd1e95b581 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.175729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334142eb-e56b-4971-b89c-cedd1e95b581 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.175729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334142eb-e56b-4971-b89c-cedd1e95b581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.190773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-334142eb-e56b-4971-b89c-cedd1e95b581 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:33.192111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-334142eb-e56b-4971-b89c-cedd1e95b581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:33.203921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6416d-f1b6-445e-9298-9796df354aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.204453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6416d-f1b6-445e-9298-9796df354aad None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.204725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6416d-f1b6-445e-9298-9796df354aad None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.204988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6416d-f1b6-445e-9298-9796df354aad None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.205666 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6416d-f1b6-445e-9298-9796df354aad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:33.205963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6416d-f1b6-445e-9298-9796df354aad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:33.207401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6416d-f1b6-445e-9298-9796df354aad None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:33.225551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 422/837] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:33.225551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6416d-f1b6-445e-9298-9796df354aad None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.265303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 416/838] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:33.266150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334142eb-e56b-4971-b89c-cedd1e95b581 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.329367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:33.347828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6416d-f1b6-445e-9298-9796df354aad None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.359176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:33.373590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334142eb-e56b-4971-b89c-cedd1e95b581 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.381823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804c7b4-56b3-4741-a782-e1709313a74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.382444 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804c7b4-56b3-4741-a782-e1709313a74e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.382715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804c7b4-56b3-4741-a782-e1709313a74e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.383002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804c7b4-56b3-4741-a782-e1709313a74e None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.383629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804c7b4-56b3-4741-a782-e1709313a74e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:33.383925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804c7b4-56b3-4741-a782-e1709313a74e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:33.385269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804c7b4-56b3-4741-a782-e1709313a74e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:33.395980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 423/839] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:33.396762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804c7b4-56b3-4741-a782-e1709313a74e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.413063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff4340-bcc4-4d37-9c47-3008a247c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.414305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff4340-bcc4-4d37-9c47-3008a247c88d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.414583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff4340-bcc4-4d37-9c47-3008a247c88d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.414857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff4340-bcc4-4d37-9c47-3008a247c88d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.415620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff4340-bcc4-4d37-9c47-3008a247c88d None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=904f7f036565419b9fdecc5e47acb45c) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:33.415946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff4340-bcc4-4d37-9c47-3008a247c88d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:33.417375 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff4340-bcc4-4d37-9c47-3008a247c88d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:33.423152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 417/840] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/role_assignments?user.id=904f7f036565419b9fdecc5e47acb45c&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=8417343e9ef64991bcf51bb9692da102 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:33.424360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345fd38-d3be-41fe-82bb-1e71b3122d81 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.424650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345fd38-d3be-41fe-82bb-1e71b3122d81 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.424929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345fd38-d3be-41fe-82bb-1e71b3122d81 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.426108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 418/841] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:33.430452 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345fd38-d3be-41fe-82bb-1e71b3122d81 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.502623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804c7b4-56b3-4741-a782-e1709313a74e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.542167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba83fa-b083-457f-a891-42e3fcdcb3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.542813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba83fa-b083-457f-a891-42e3fcdcb3ca None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.543132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba83fa-b083-457f-a891-42e3fcdcb3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.543401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba83fa-b083-457f-a891-42e3fcdcb3ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.548189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345fd38-d3be-41fe-82bb-1e71b3122d81 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.555883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba83fa-b083-457f-a891-42e3fcdcb3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-6aba83fa-b083-457f-a891-42e3fcdcb3ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:33.562339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:33.563543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:33.564387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 424/842] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:33.565074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a8ab4-5fa5-4f89-8399-9c9ba78790fe None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.565264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a8ab4-5fa5-4f89-8399-9c9ba78790fe None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.565529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a8ab4-5fa5-4f89-8399-9c9ba78790fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.575478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:33.588360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-029a8ab4-5fa5-4f89-8399-9c9ba78790fe None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:33.589526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-029a8ab4-5fa5-4f89-8399-9c9ba78790fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:33.600074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365a59a-7c42-48c4-9309-b01eb3adc393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.602252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365a59a-7c42-48c4-9309-b01eb3adc393 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.602252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365a59a-7c42-48c4-9309-b01eb3adc393 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.602252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365a59a-7c42-48c4-9309-b01eb3adc393 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.609629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365a59a-7c42-48c4-9309-b01eb3adc393 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:33.611093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365a59a-7c42-48c4-9309-b01eb3adc393 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6365a59a-7c42-48c4-9309-b01eb3adc393 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:33.623606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:33.624540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:33.625453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 419/843] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:33.626031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365a59a-7c42-48c4-9309-b01eb3adc393 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.645216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:33.675499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 425/844] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:33.675499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a8ab4-5fa5-4f89-8399-9c9ba78790fe None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.744682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365a59a-7c42-48c4-9309-b01eb3adc393 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.778841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a8ab4-5fa5-4f89-8399-9c9ba78790fe None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.781193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f7e15-6387-4d8c-82ea-42d84844e082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.781803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f7e15-6387-4d8c-82ea-42d84844e082 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.782083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f7e15-6387-4d8c-82ea-42d84844e082 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.782355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f7e15-6387-4d8c-82ea-42d84844e082 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.783047 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f7e15-6387-4d8c-82ea-42d84844e082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:33.783353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f7e15-6387-4d8c-82ea-42d84844e082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:33.784718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f7e15-6387-4d8c-82ea-42d84844e082 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:33.793644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 420/845] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:33.793644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f7e15-6387-4d8c-82ea-42d84844e082 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.818103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d2be3-748a-4875-9a08-62ba224f0b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.818710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d2be3-748a-4875-9a08-62ba224f0b76 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.819009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d2be3-748a-4875-9a08-62ba224f0b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.819332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d2be3-748a-4875-9a08-62ba224f0b76 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.820618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d2be3-748a-4875-9a08-62ba224f0b76 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:33.820917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d2be3-748a-4875-9a08-62ba224f0b76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:33.822369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d2be3-748a-4875-9a08-62ba224f0b76 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:33.823932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:33.845669 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 426/846] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:33.846325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d2be3-748a-4875-9a08-62ba224f0b76 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.875523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:33.902561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f7e15-6387-4d8c-82ea-42d84844e082 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.943189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab586-7bdd-41c0-ac04-6029a2810448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.943928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab586-7bdd-41c0-ac04-6029a2810448 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.944219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab586-7bdd-41c0-ac04-6029a2810448 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.944587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab586-7bdd-41c0-ac04-6029a2810448 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:33.945600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ab586-7bdd-41c0-ac04-6029a2810448 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:33.946969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ab586-7bdd-41c0-ac04-6029a2810448 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:33.952506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d2be3-748a-4875-9a08-62ba224f0b76 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ff4ab586-7bdd-41c0-ac04-6029a2810448 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:33.955582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:33.956541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:33.962668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 421/847] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:33.963703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab586-7bdd-41c0-ac04-6029a2810448 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:33.992471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbe356-e459-4969-8d6b-a14c2ac4e7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:33.993142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbe356-e459-4969-8d6b-a14c2ac4e7ac None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:33.993728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbe356-e459-4969-8d6b-a14c2ac4e7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:33.994007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbe356-e459-4969-8d6b-a14c2ac4e7ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.004684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbe356-e459-4969-8d6b-a14c2ac4e7ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.006517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbe356-e459-4969-8d6b-a14c2ac4e7ac None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-7dfbe356-e459-4969-8d6b-a14c2ac4e7ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:34.023578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:34.024798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:34.025895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:34.025895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:34.025895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:34.029611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 427/848] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:34.030301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbe356-e459-4969-8d6b-a14c2ac4e7ac None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.067374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab586-7bdd-41c0-ac04-6029a2810448 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:34.078415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:34.102841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f02b7-41a4-4c98-9346-2649f43ca0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:34.103539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f02b7-41a4-4c98-9346-2649f43ca0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:34.103849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f02b7-41a4-4c98-9346-2649f43ca0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:34.104117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f02b7-41a4-4c98-9346-2649f43ca0a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.112599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:34.136750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbe356-e459-4969-8d6b-a14c2ac4e7ac None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:34.169636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb89fa-75b6-4c83-b239-0fab8f6ddd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:34.170242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb89fa-75b6-4c83-b239-0fab8f6ddd46 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:34.170527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb89fa-75b6-4c83-b239-0fab8f6ddd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:34.170803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb89fa-75b6-4c83-b239-0fab8f6ddd46 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.171467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb89fa-75b6-4c83-b239-0fab8f6ddd46 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:34.171760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb89fa-75b6-4c83-b239-0fab8f6ddd46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.173146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb89fa-75b6-4c83-b239-0fab8f6ddd46 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.174146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f02b7-41a4-4c98-9346-2649f43ca0a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.175600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f02b7-41a4-4c98-9346-2649f43ca0a3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.182453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 428/849] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:34.182453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb89fa-75b6-4c83-b239-0fab8f6ddd46 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.282605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb89fa-75b6-4c83-b239-0fab8f6ddd46 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:34.284987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 422/850] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:51:34.285965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f02b7-41a4-4c98-9346-2649f43ca0a3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.296832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:34.321357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bcbed-414a-41b7-ae87-3082c3114be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:34.321953 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bcbed-414a-41b7-ae87-3082c3114be4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:34.322224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bcbed-414a-41b7-ae87-3082c3114be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:34.322488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bcbed-414a-41b7-ae87-3082c3114be4 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.323148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bcbed-414a-41b7-ae87-3082c3114be4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:34.323446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bcbed-414a-41b7-ae87-3082c3114be4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.324794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bcbed-414a-41b7-ae87-3082c3114be4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.330567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:34.342300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 429/851] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:34.342300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bcbed-414a-41b7-ae87-3082c3114be4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.406273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f02b7-41a4-4c98-9346-2649f43ca0a3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:34.444627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1058bc3-2452-421a-98fa-6a5b98f3d168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:34.445235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058bc3-2452-421a-98fa-6a5b98f3d168 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:34.445551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058bc3-2452-421a-98fa-6a5b98f3d168 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:34.445830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058bc3-2452-421a-98fa-6a5b98f3d168 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.446454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1058bc3-2452-421a-98fa-6a5b98f3d168 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:34.446755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1058bc3-2452-421a-98fa-6a5b98f3d168 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.448159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1058bc3-2452-421a-98fa-6a5b98f3d168 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.448890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bcbed-414a-41b7-ae87-3082c3114be4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:34.462996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 423/852] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:34.462996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1058bc3-2452-421a-98fa-6a5b98f3d168 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.496000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf178b-ee98-43af-91c6-bddc7b25b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:34.496859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf178b-ee98-43af-91c6-bddc7b25b852 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:34.497203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf178b-ee98-43af-91c6-bddc7b25b852 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:34.497635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf178b-ee98-43af-91c6-bddc7b25b852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.509472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf178b-ee98-43af-91c6-bddc7b25b852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.511297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf178b-ee98-43af-91c6-bddc7b25b852 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-4fdf178b-ee98-43af-91c6-bddc7b25b852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:34.530192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:34.531421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:34.532512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:34.532512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:34.532512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:34.539060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 430/853] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:34.540202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf178b-ee98-43af-91c6-bddc7b25b852 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.568500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:34.582564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1058bc3-2452-421a-98fa-6a5b98f3d168 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:34.584432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:34.623135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b2047-89c3-4d9c-a6ae-5a985191103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:34.624151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b2047-89c3-4d9c-a6ae-5a985191103f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:34.624522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b2047-89c3-4d9c-a6ae-5a985191103f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:34.624891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b2047-89c3-4d9c-a6ae-5a985191103f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.633781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b2047-89c3-4d9c-a6ae-5a985191103f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.635461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b2047-89c3-4d9c-a6ae-5a985191103f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f61b2047-89c3-4d9c-a6ae-5a985191103f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:34.649798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:34.650805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 09:51:34.651823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:34.653966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 424/854] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:34.660918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b2047-89c3-4d9c-a6ae-5a985191103f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.687702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf178b-ee98-43af-91c6-bddc7b25b852 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:34.723312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44861a-de9f-49d9-a7bb-63170b97ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:34.724058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44861a-de9f-49d9-a7bb-63170b97ab31 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:34.724390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44861a-de9f-49d9-a7bb-63170b97ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:34.724752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44861a-de9f-49d9-a7bb-63170b97ab31 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.725450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44861a-de9f-49d9-a7bb-63170b97ab31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:34.725877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44861a-de9f-49d9-a7bb-63170b97ab31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.727900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44861a-de9f-49d9-a7bb-63170b97ab31 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.745791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 431/855] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:34.746767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44861a-de9f-49d9-a7bb-63170b97ab31 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.791497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b2047-89c3-4d9c-a6ae-5a985191103f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:34.828363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f8dfa-5771-472d-ae76-6e4d62826cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:34.829070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f8dfa-5771-472d-ae76-6e4d62826cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:34.829481 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f8dfa-5771-472d-ae76-6e4d62826cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:34.829812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f8dfa-5771-472d-ae76-6e4d62826cd6 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.830638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f8dfa-5771-472d-ae76-6e4d62826cd6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:34.831054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f8dfa-5771-472d-ae76-6e4d62826cd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.832549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f8dfa-5771-472d-ae76-6e4d62826cd6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.840799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:34.855345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 425/856] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:34.860630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f8dfa-5771-472d-ae76-6e4d62826cd6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.868120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44861a-de9f-49d9-a7bb-63170b97ab31 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:34.890572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:34.918099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b7c4-9e71-49fb-93c8-f74253d22d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:34.919460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b7c4-9e71-49fb-93c8-f74253d22d47 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:34.919914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b7c4-9e71-49fb-93c8-f74253d22d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:34.920386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b7c4-9e71-49fb-93c8-f74253d22d47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:34.930248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27b7c4-9e71-49fb-93c8-f74253d22d47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:34.932499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27b7c4-9e71-49fb-93c8-f74253d22d47 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-6d27b7c4-9e71-49fb-93c8-f74253d22d47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:34.954563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:34.955718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:34.956825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:34.956825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:34.956825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:34.956825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:34.956825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:34.956825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:34.956825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:34.956825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:34.958812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 432/857] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:34.965148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b7c4-9e71-49fb-93c8-f74253d22d47 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:34.991729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f8dfa-5771-472d-ae76-6e4d62826cd6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:35.044998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642a57-4ad7-4d48-8665-81de2ae74d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:35.046155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642a57-4ad7-4d48-8665-81de2ae74d93 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.046582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642a57-4ad7-4d48-8665-81de2ae74d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.047434 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642a57-4ad7-4d48-8665-81de2ae74d93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.091333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b7c4-9e71-49fb-93c8-f74253d22d47 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:35.126642 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dd569-d6fc-477c-9d7c-422d2278595f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:35.127445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dd569-d6fc-477c-9d7c-422d2278595f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.127815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dd569-d6fc-477c-9d7c-422d2278595f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.128263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dd569-d6fc-477c-9d7c-422d2278595f None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.129053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dd569-d6fc-477c-9d7c-422d2278595f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:35.129531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dd569-d6fc-477c-9d7c-422d2278595f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:35.131122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dd569-d6fc-477c-9d7c-422d2278595f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:35.142382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:35.145691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642a57-4ad7-4d48-8665-81de2ae74d93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:35.146510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 433/858] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:35.148632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642a57-4ad7-4d48-8665-81de2ae74d93 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:35.152685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dd569-d6fc-477c-9d7c-422d2278595f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:35.173495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:35.255636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dd569-d6fc-477c-9d7c-422d2278595f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:35.262670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 426/859] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:34 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 09:51:35.269745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642a57-4ad7-4d48-8665-81de2ae74d93 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:35.295290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd9924-c1fc-4719-b17e-5a56f45390ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:35.296088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd9924-c1fc-4719-b17e-5a56f45390ef None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.296369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd9924-c1fc-4719-b17e-5a56f45390ef None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.296637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd9924-c1fc-4719-b17e-5a56f45390ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.297600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd9924-c1fc-4719-b17e-5a56f45390ef None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=36f9577e047a406480fadc4b9f8c6234) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:35.297904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd9924-c1fc-4719-b17e-5a56f45390ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:35.300058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd9924-c1fc-4719-b17e-5a56f45390ef None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:35.311053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 434/860] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:35 2022] GET /identity/v3/role_assignments?user.id=36f9577e047a406480fadc4b9f8c6234&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:35.388918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642a57-4ad7-4d48-8665-81de2ae74d93 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:35.417114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:35.432952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c16b6-f9c8-48c7-b82c-153b790dd2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:35.433599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c16b6-f9c8-48c7-b82c-153b790dd2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.433874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c16b6-f9c8-48c7-b82c-153b790dd2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.434142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c16b6-f9c8-48c7-b82c-153b790dd2b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.445870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c16b6-f9c8-48c7-b82c-153b790dd2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:35.447355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c16b6-f9c8-48c7-b82c-153b790dd2b1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-631c16b6-f9c8-48c7-b82c-153b790dd2b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:35.463242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:35.464245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:35.465133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:35.465133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:35.465133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:35.465133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 427/861] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:35.470323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd9924-c1fc-4719-b17e-5a56f45390ef None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:35.488033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643ce5d-fe8a-47ef-93e4-c422872beb1b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.488604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643ce5d-fe8a-47ef-93e4-c422872beb1b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.488877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643ce5d-fe8a-47ef-93e4-c422872beb1b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.490754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 428/862] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:35.496629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b81583-8d82-4200-90ad-b876e847928d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.496905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b81583-8d82-4200-90ad-b876e847928d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.497173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b81583-8d82-4200-90ad-b876e847928d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.517298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-19b81583-8d82-4200-90ad-b876e847928d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:35.518749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-19b81583-8d82-4200-90ad-b876e847928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:35.556547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:35.601962 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 429/863] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:35.603592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd9924-c1fc-4719-b17e-5a56f45390ef None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:35.626140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b81583-8d82-4200-90ad-b876e847928d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:35.644105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098df97-cb7a-45f2-80f7-d9801d5434e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:35.644731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098df97-cb7a-45f2-80f7-d9801d5434e9 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.645011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098df97-cb7a-45f2-80f7-d9801d5434e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.645281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098df97-cb7a-45f2-80f7-d9801d5434e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.645935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098df97-cb7a-45f2-80f7-d9801d5434e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:35.646235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098df97-cb7a-45f2-80f7-d9801d5434e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:35.647849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098df97-cb7a-45f2-80f7-d9801d5434e9 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:35.661417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 435/864] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:35.666788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098df97-cb7a-45f2-80f7-d9801d5434e9 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:35.754579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b81583-8d82-4200-90ad-b876e847928d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:35.756572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:35.793385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098df97-cb7a-45f2-80f7-d9801d5434e9 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:35.797349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec6f05-26e0-4231-98d1-9d747452a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:35.797989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec6f05-26e0-4231-98d1-9d747452a532 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.798265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec6f05-26e0-4231-98d1-9d747452a532 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.798535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec6f05-26e0-4231-98d1-9d747452a532 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.799167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec6f05-26e0-4231-98d1-9d747452a532 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:35.800532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec6f05-26e0-4231-98d1-9d747452a532 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-d2ec6f05-26e0-4231-98d1-9d747452a532 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:35.811542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:35.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:35.813747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 430/865] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Aug 10 09:51:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:35.818406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec6f05-26e0-4231-98d1-9d747452a532 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:35.827694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:35.831183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da181255-ae18-4ec6-b578-a5a53e0bf21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:35.831880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da181255-ae18-4ec6-b578-a5a53e0bf21f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.832221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da181255-ae18-4ec6-b578-a5a53e0bf21f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.832547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da181255-ae18-4ec6-b578-a5a53e0bf21f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.839800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da181255-ae18-4ec6-b578-a5a53e0bf21f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:35.841225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da181255-ae18-4ec6-b578-a5a53e0bf21f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-da181255-ae18-4ec6-b578-a5a53e0bf21f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:35.852941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:35.854684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:35.856627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:35.856627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:35.856627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:35.856627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:35.856627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:35.856627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:35.856627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:35.856627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:35.856627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 436/866] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:35.862978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da181255-ae18-4ec6-b578-a5a53e0bf21f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:35.950861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec6f05-26e0-4231-98d1-9d747452a532 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:35.965138 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da181255-ae18-4ec6-b578-a5a53e0bf21f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:35.992629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d67036-79af-4aec-bcb9-e24c28089ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:35.993230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d67036-79af-4aec-bcb9-e24c28089ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:35.993756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d67036-79af-4aec-bcb9-e24c28089ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:35.994031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d67036-79af-4aec-bcb9-e24c28089ff1 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:35.994715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d67036-79af-4aec-bcb9-e24c28089ff1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:35.995035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d67036-79af-4aec-bcb9-e24c28089ff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:35.996389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d67036-79af-4aec-bcb9-e24c28089ff1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:36.005786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 431/867] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:36.006657 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d67036-79af-4aec-bcb9-e24c28089ff1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:36.006838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aa95e-38b3-4ed8-bfbd-96be9dd74b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:36.007499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aa95e-38b3-4ed8-bfbd-96be9dd74b7c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:36.007772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aa95e-38b3-4ed8-bfbd-96be9dd74b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:36.008040 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aa95e-38b3-4ed8-bfbd-96be9dd74b7c None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:36.008724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aa95e-38b3-4ed8-bfbd-96be9dd74b7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:36.009025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aa95e-38b3-4ed8-bfbd-96be9dd74b7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:36.011774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aa95e-38b3-4ed8-bfbd-96be9dd74b7c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:36.026543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 437/868] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:36.032403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aa95e-38b3-4ed8-bfbd-96be9dd74b7c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:36.046062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:36.087782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:36.119182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d67036-79af-4aec-bcb9-e24c28089ff1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:36.150452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aa95e-38b3-4ed8-bfbd-96be9dd74b7c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:36.155058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ec674-e4a8-43aa-ab78-c64e2ad98d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:36.155722 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ec674-e4a8-43aa-ab78-c64e2ad98d7d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:36.155996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ec674-e4a8-43aa-ab78-c64e2ad98d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:36.156309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ec674-e4a8-43aa-ab78-c64e2ad98d7d None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:36.156991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ec674-e4a8-43aa-ab78-c64e2ad98d7d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:36.157293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ec674-e4a8-43aa-ab78-c64e2ad98d7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:36.158644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ec674-e4a8-43aa-ab78-c64e2ad98d7d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:36.171723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 432/869] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:36.191340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17240aec-5c68-4219-9692-f5b16240a3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:36.192350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17240aec-5c68-4219-9692-f5b16240a3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:36.192843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17240aec-5c68-4219-9692-f5b16240a3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:36.193303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17240aec-5c68-4219-9692-f5b16240a3b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:36.194239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17240aec-5c68-4219-9692-f5b16240a3b7 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=af29f3451f654f53ac98239a2b28d128) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:36.194800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17240aec-5c68-4219-9692-f5b16240a3b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:36.196482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17240aec-5c68-4219-9692-f5b16240a3b7 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:36.203921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 438/870] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:36 2022] GET /identity/v3/role_assignments?user.id=af29f3451f654f53ac98239a2b28d128&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=8417343e9ef64991bcf51bb9692da102 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:36.591003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f2361-0749-463e-82d2-f749685e6836 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:36.591395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f2361-0749-463e-82d2-f749685e6836 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:36.591678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f2361-0749-463e-82d2-f749685e6836 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:36.593229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 433/871] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:36.598912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3622361-23b6-41a7-b67c-94a217d1aeb1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:36.599237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3622361-23b6-41a7-b67c-94a217d1aeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:36.599485 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3622361-23b6-41a7-b67c-94a217d1aeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:36.619376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f3622361-23b6-41a7-b67c-94a217d1aeb1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:36.620554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f3622361-23b6-41a7-b67c-94a217d1aeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:36.690773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 439/872] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:36.713451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f2361-0749-463e-82d2-f749685e6836 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:36.809695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:36.818275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f2361-0749-463e-82d2-f749685e6836 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:36.852719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0012f-c2dd-488b-843a-3f1e0eb46ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:36.853321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0012f-c2dd-488b-843a-3f1e0eb46ab6 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:36.853591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0012f-c2dd-488b-843a-3f1e0eb46ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:36.853856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0012f-c2dd-488b-843a-3f1e0eb46ab6 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:36.854560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0012f-c2dd-488b-843a-3f1e0eb46ab6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:36.855945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0012f-c2dd-488b-843a-3f1e0eb46ab6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:36.875789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 434/873] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:51:36 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:37.529829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8165159-d423-4647-a2b6-cb1dc81a55bd None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:37.532580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8165159-d423-4647-a2b6-cb1dc81a55bd None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:37.532580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8165159-d423-4647-a2b6-cb1dc81a55bd None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:37.533196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 440/874] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:37.538576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97586e-ad9b-4126-baa5-4386e783f54a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:37.539426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97586e-ad9b-4126-baa5-4386e783f54a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:37.539913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97586e-ad9b-4126-baa5-4386e783f54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:37.558656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4b97586e-ad9b-4126-baa5-4386e783f54a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:37.560175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4b97586e-ad9b-4126-baa5-4386e783f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:37.639527 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 435/875] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:37 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:37.664131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8165159-d423-4647-a2b6-cb1dc81a55bd None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:37.709240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:37.772931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8165159-d423-4647-a2b6-cb1dc81a55bd None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:37.808345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caf22f-926b-4d48-8697-a9b91f3fb7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:37.809437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caf22f-926b-4d48-8697-a9b91f3fb7ba None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:37.809916 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caf22f-926b-4d48-8697-a9b91f3fb7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:37.810385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caf22f-926b-4d48-8697-a9b91f3fb7ba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:37.821656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caf22f-926b-4d48-8697-a9b91f3fb7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-98caf22f-926b-4d48-8697-a9b91f3fb7ba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:37.826529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:37.827481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:51:37.828317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:37.835131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 441/876] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 10 09:51:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:37.837742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97586e-ad9b-4126-baa5-4386e783f54a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:37.923384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:37.943402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97586e-ad9b-4126-baa5-4386e783f54a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:37.985688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb23bd-01a0-4e76-a732-ccf54c06f5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:37.986295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb23bd-01a0-4e76-a732-ccf54c06f5fe None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:37.986567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb23bd-01a0-4e76-a732-ccf54c06f5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:37.986831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb23bd-01a0-4e76-a732-ccf54c06f5fe None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:37.987525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb23bd-01a0-4e76-a732-ccf54c06f5fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:37.987816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb23bd-01a0-4e76-a732-ccf54c06f5fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:37.989159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb23bd-01a0-4e76-a732-ccf54c06f5fe None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:37.998455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 436/877] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 10 09:51:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:38.004207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caf22f-926b-4d48-8697-a9b91f3fb7ba None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:38.111364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:38.114242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caf22f-926b-4d48-8697-a9b91f3fb7ba None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:38.151668 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f0436-7a80-4f9e-8889-acf2e437eb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:38.152296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f0436-7a80-4f9e-8889-acf2e437eb6a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.152568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f0436-7a80-4f9e-8889-acf2e437eb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.152836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f0436-7a80-4f9e-8889-acf2e437eb6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.163005 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f0436-7a80-4f9e-8889-acf2e437eb6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:38.165026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f0436-7a80-4f9e-8889-acf2e437eb6a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-fd0f0436-7a80-4f9e-8889-acf2e437eb6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:38.180371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:38.181356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:38.182667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:38.182667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:38.182667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:38.182667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 442/878] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:38.188762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb23bd-01a0-4e76-a732-ccf54c06f5fe None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:38.234778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c519b85-5e6c-47af-a219-5daafc4b8c36 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.235132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c519b85-5e6c-47af-a219-5daafc4b8c36 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.235370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c519b85-5e6c-47af-a219-5daafc4b8c36 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.236944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 443/879] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:38.242282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12d99b-e664-4444-9e03-0811d91194ee None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.242558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12d99b-e664-4444-9e03-0811d91194ee None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.242827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12d99b-e664-4444-9e03-0811d91194ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.262659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-5c12d99b-e664-4444-9e03-0811d91194ee None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:38.263856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5c12d99b-e664-4444-9e03-0811d91194ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:38.300182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb23bd-01a0-4e76-a732-ccf54c06f5fe None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:38.329855 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 444/880] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:38 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:38.331096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863cb58-3927-48b6-a9ef-452ed943fff9 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.331376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863cb58-3927-48b6-a9ef-452ed943fff9 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.331639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863cb58-3927-48b6-a9ef-452ed943fff9 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.333016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 445/881] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:38.338487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f04ea-b98f-491d-b4bb-c6c123e5c70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:38.338621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14888fdd-f3d8-4b8a-9fe5-222fc4a29056 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.338871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14888fdd-f3d8-4b8a-9fe5-222fc4a29056 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.339165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14888fdd-f3d8-4b8a-9fe5-222fc4a29056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.339891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f04ea-b98f-491d-b4bb-c6c123e5c70b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.340381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f04ea-b98f-491d-b4bb-c6c123e5c70b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.340845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f04ea-b98f-491d-b4bb-c6c123e5c70b None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.341842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f04ea-b98f-491d-b4bb-c6c123e5c70b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:38.342395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f04ea-b98f-491d-b4bb-c6c123e5c70b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:38.344097 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f04ea-b98f-491d-b4bb-c6c123e5c70b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:38.357150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 437/882] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:38.357988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-14888fdd-f3d8-4b8a-9fe5-222fc4a29056 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:38.358617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f04ea-b98f-491d-b4bb-c6c123e5c70b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:38.359348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-14888fdd-f3d8-4b8a-9fe5-222fc4a29056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:38.361582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:38.436605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 446/883] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:38 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:38.438222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14888fdd-f3d8-4b8a-9fe5-222fc4a29056 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:38.459045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:38.468747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f04ea-b98f-491d-b4bb-c6c123e5c70b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:38.504899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dc8b0-7f34-4a0b-9d91-7a109aef8c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:38.505504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dc8b0-7f34-4a0b-9d91-7a109aef8c18 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.505779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dc8b0-7f34-4a0b-9d91-7a109aef8c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.506048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dc8b0-7f34-4a0b-9d91-7a109aef8c18 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.506684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7dc8b0-7f34-4a0b-9d91-7a109aef8c18 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:38.508364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7dc8b0-7f34-4a0b-9d91-7a109aef8c18 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:38.525821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 438/884] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:51:38 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:38.526986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dc8b0-7f34-4a0b-9d91-7a109aef8c18 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:38.573408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14888fdd-f3d8-4b8a-9fe5-222fc4a29056 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:38.611549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:38.627359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4195e7-2308-4515-8f08-b9e638167d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:38.628244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4195e7-2308-4515-8f08-b9e638167d86 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.628617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4195e7-2308-4515-8f08-b9e638167d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.629055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4195e7-2308-4515-8f08-b9e638167d86 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.638422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4195e7-2308-4515-8f08-b9e638167d86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:38.639908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4195e7-2308-4515-8f08-b9e638167d86 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-ba4195e7-2308-4515-8f08-b9e638167d86 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:38.654500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:38.655712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:38.656829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:38.662417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 447/885] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:51:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:38.664050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4195e7-2308-4515-8f08-b9e638167d86 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:38.670891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dc8b0-7f34-4a0b-9d91-7a109aef8c18 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:38.716433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74132da0-bb28-4320-8da7-861356bf1741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:38.716994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74132da0-bb28-4320-8da7-861356bf1741 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.717270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74132da0-bb28-4320-8da7-861356bf1741 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.717538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74132da0-bb28-4320-8da7-861356bf1741 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.718122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74132da0-bb28-4320-8da7-861356bf1741 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:38.719523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74132da0-bb28-4320-8da7-861356bf1741 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-74132da0-bb28-4320-8da7-861356bf1741 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:38.729043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:38.730034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:38.730849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 439/886] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:38.736358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74132da0-bb28-4320-8da7-861356bf1741 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:38.754754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:38.793649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4195e7-2308-4515-8f08-b9e638167d86 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:38.830276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32025f6d-8115-4e4c-bf1b-f27e4dc9ea35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:38.830877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32025f6d-8115-4e4c-bf1b-f27e4dc9ea35 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.831193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32025f6d-8115-4e4c-bf1b-f27e4dc9ea35 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.831459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32025f6d-8115-4e4c-bf1b-f27e4dc9ea35 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.832271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32025f6d-8115-4e4c-bf1b-f27e4dc9ea35 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:38.832572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32025f6d-8115-4e4c-bf1b-f27e4dc9ea35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:38.834087 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32025f6d-8115-4e4c-bf1b-f27e4dc9ea35 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:38.842965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74132da0-bb28-4320-8da7-861356bf1741 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:38.858045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 448/887] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 10 09:51:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:38.859135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32025f6d-8115-4e4c-bf1b-f27e4dc9ea35 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:38.877866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:38.885996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7677ba9-42ec-4e98-89fb-f0a4fa373795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:38.886893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7677ba9-42ec-4e98-89fb-f0a4fa373795 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:38.887397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7677ba9-42ec-4e98-89fb-f0a4fa373795 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:38.887949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7677ba9-42ec-4e98-89fb-f0a4fa373795 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:38.889104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7677ba9-42ec-4e98-89fb-f0a4fa373795 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:38.889609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7677ba9-42ec-4e98-89fb-f0a4fa373795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:38.891514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7677ba9-42ec-4e98-89fb-f0a4fa373795 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:38.898855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 440/888] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:38.906610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7677ba9-42ec-4e98-89fb-f0a4fa373795 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:38.968861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32025f6d-8115-4e4c-bf1b-f27e4dc9ea35 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.002572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:39.008781 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f64cbc-72c2-4d63-b431-621c150dd691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:39.009514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f64cbc-72c2-4d63-b431-621c150dd691 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.009796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f64cbc-72c2-4d63-b431-621c150dd691 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.010073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f64cbc-72c2-4d63-b431-621c150dd691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.035667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7677ba9-42ec-4e98-89fb-f0a4fa373795 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.075680 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c21a77-5e84-459f-ae72-54ec632a5079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:39.076292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c21a77-5e84-459f-ae72-54ec632a5079 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.076570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c21a77-5e84-459f-ae72-54ec632a5079 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.076841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c21a77-5e84-459f-ae72-54ec632a5079 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.077544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c21a77-5e84-459f-ae72-54ec632a5079 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:39.077843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c21a77-5e84-459f-ae72-54ec632a5079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:39.079245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c21a77-5e84-459f-ae72-54ec632a5079 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:39.083587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 441/889] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:39.084112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f64cbc-72c2-4d63-b431-621c150dd691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:39.084736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2198f-1d0b-4a23-a67e-685842278d30 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.085546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2198f-1d0b-4a23-a67e-685842278d30 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.085823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2198f-1d0b-4a23-a67e-685842278d30 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.086789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f64cbc-72c2-4d63-b431-621c150dd691 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:39.087351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 442/890] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:39.092771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd394052-1194-490f-ad36-b639a2f49ebb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.093299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd394052-1194-490f-ad36-b639a2f49ebb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.093732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd394052-1194-490f-ad36-b639a2f49ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.114502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dd394052-1194-490f-ad36-b639a2f49ebb None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:39.116381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dd394052-1194-490f-ad36-b639a2f49ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:39.191311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:39.209637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 443/891] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:39.210676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 449/892] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:38 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 09:51:39.215892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd394052-1194-490f-ad36-b639a2f49ebb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:39.233295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f64cbc-72c2-4d63-b431-621c150dd691 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:39.269031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:39.334183 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd394052-1194-490f-ad36-b639a2f49ebb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.356852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f64cbc-72c2-4d63-b431-621c150dd691 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.380636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e962-15fb-4e0f-b3da-a3fef83076a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:39.381725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e962-15fb-4e0f-b3da-a3fef83076a3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.382578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e962-15fb-4e0f-b3da-a3fef83076a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.383219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e962-15fb-4e0f-b3da-a3fef83076a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.397421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1032997-2620-4bd7-b9eb-0f74c90d5dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:39.398836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1032997-2620-4bd7-b9eb-0f74c90d5dda None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.399318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1032997-2620-4bd7-b9eb-0f74c90d5dda None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.399962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1032997-2620-4bd7-b9eb-0f74c90d5dda None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.400837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e962-15fb-4e0f-b3da-a3fef83076a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:39.402477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e962-15fb-4e0f-b3da-a3fef83076a3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:39.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1032997-2620-4bd7-b9eb-0f74c90d5dda None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:39.404836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1032997-2620-4bd7-b9eb-0f74c90d5dda None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-e1032997-2620-4bd7-b9eb-0f74c90d5dda None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:39.422070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:39.423074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 09:51:39.423930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:39.428218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 450/893] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:51:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e334e962-15fb-4e0f-b3da-a3fef83076a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:39.428583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:39.429536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:39.430408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:39.430408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:39.430408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:39.430573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1032997-2620-4bd7-b9eb-0f74c90d5dda None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:39.432310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 444/894] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:39.439930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e962-15fb-4e0f-b3da-a3fef83076a3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:39.487713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:39.533602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:39.548729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1032997-2620-4bd7-b9eb-0f74c90d5dda None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.580009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e962-15fb-4e0f-b3da-a3fef83076a3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.589455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d4bc9-0eb1-4ce0-868f-3af1036cf52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:39.590226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d4bc9-0eb1-4ce0-868f-3af1036cf52d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.590610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d4bc9-0eb1-4ce0-868f-3af1036cf52d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.591007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d4bc9-0eb1-4ce0-868f-3af1036cf52d None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.591744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d4bc9-0eb1-4ce0-868f-3af1036cf52d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:39.592205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d4bc9-0eb1-4ce0-868f-3af1036cf52d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:39.593730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d4bc9-0eb1-4ce0-868f-3af1036cf52d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:39.601353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 451/895] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:51:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:39.606774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d4bc9-0eb1-4ce0-868f-3af1036cf52d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:39.623217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74dc5f-4cad-417f-a87a-8f93351b1780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:39.624063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74dc5f-4cad-417f-a87a-8f93351b1780 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.624337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74dc5f-4cad-417f-a87a-8f93351b1780 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.625072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74dc5f-4cad-417f-a87a-8f93351b1780 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.625972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74dc5f-4cad-417f-a87a-8f93351b1780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:39.626280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74dc5f-4cad-417f-a87a-8f93351b1780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:39.628256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74dc5f-4cad-417f-a87a-8f93351b1780 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:39.641404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 445/896] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:39.647033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74dc5f-4cad-417f-a87a-8f93351b1780 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:39.735613 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d4bc9-0eb1-4ce0-868f-3af1036cf52d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.754734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:39.763802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74dc5f-4cad-417f-a87a-8f93351b1780 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.776389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab80d19-1c78-492b-a4d4-f50dbae5ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:39.777127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab80d19-1c78-492b-a4d4-f50dbae5ea00 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.777561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab80d19-1c78-492b-a4d4-f50dbae5ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.778026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab80d19-1c78-492b-a4d4-f50dbae5ea00 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.778759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab80d19-1c78-492b-a4d4-f50dbae5ea00 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:39.779195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab80d19-1c78-492b-a4d4-f50dbae5ea00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:39.780865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab80d19-1c78-492b-a4d4-f50dbae5ea00 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:39.791313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 452/897] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:51:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:39.792549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab80d19-1c78-492b-a4d4-f50dbae5ea00 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:39.803560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d233f82-38d8-46c4-bbf2-a3d00f72fca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:39.804704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d233f82-38d8-46c4-bbf2-a3d00f72fca8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.804980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d233f82-38d8-46c4-bbf2-a3d00f72fca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.805246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d233f82-38d8-46c4-bbf2-a3d00f72fca8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.814091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:39.816082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d233f82-38d8-46c4-bbf2-a3d00f72fca8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:39.817537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d233f82-38d8-46c4-bbf2-a3d00f72fca8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-4d233f82-38d8-46c4-bbf2-a3d00f72fca8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:39.834889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:39.836944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:39.839037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:39.839037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:39.839037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:39.839037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:39.839037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:39.839037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:39.839037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:39.839037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:39.842525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 446/898] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:39.842525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d233f82-38d8-46c4-bbf2-a3d00f72fca8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:39.917967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab80d19-1c78-492b-a4d4-f50dbae5ea00 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.962453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d233f82-38d8-46c4-bbf2-a3d00f72fca8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:39.963467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3504adb-ec25-4c9a-b689-e7ec9bb53ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:39.964314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3504adb-ec25-4c9a-b689-e7ec9bb53ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:39.964595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3504adb-ec25-4c9a-b689-e7ec9bb53ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:39.964864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3504adb-ec25-4c9a-b689-e7ec9bb53ff0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:39.966371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3504adb-ec25-4c9a-b689-e7ec9bb53ff0 None admin] RBAC: Adding query filter params (interface=public, service_id=4804b573c46641d2abb3e736907ab3c8, region_id=RegionOne) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:39.966676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3504adb-ec25-4c9a-b689-e7ec9bb53ff0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:39.968053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3504adb-ec25-4c9a-b689-e7ec9bb53ff0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:39.974663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 453/899] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 10 09:51:39 2022] GET /identity/v3/endpoints?service_id=4804b573c46641d2abb3e736907ab3c8&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:39.980057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3504adb-ec25-4c9a-b689-e7ec9bb53ff0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:40.004570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a130641-45fa-4d5d-9cb1-e0aaa76e3056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:40.005511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a130641-45fa-4d5d-9cb1-e0aaa76e3056 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:40.005784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a130641-45fa-4d5d-9cb1-e0aaa76e3056 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:40.006059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a130641-45fa-4d5d-9cb1-e0aaa76e3056 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:40.006749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a130641-45fa-4d5d-9cb1-e0aaa76e3056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:40.007679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a130641-45fa-4d5d-9cb1-e0aaa76e3056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:40.009602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a130641-45fa-4d5d-9cb1-e0aaa76e3056 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:40.025314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 447/900] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:40.031686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a130641-45fa-4d5d-9cb1-e0aaa76e3056 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:40.053621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:40.083637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:40.107915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3504adb-ec25-4c9a-b689-e7ec9bb53ff0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:40.145374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b54e19-500e-45a0-acb6-5b798f207027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:40.146012 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b54e19-500e-45a0-acb6-5b798f207027 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:40.146291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b54e19-500e-45a0-acb6-5b798f207027 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:40.146562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b54e19-500e-45a0-acb6-5b798f207027 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:40.147475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b54e19-500e-45a0-acb6-5b798f207027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:40.148712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b54e19-500e-45a0-acb6-5b798f207027 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:40.163567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 454/901] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 10 09:51:39 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:51:40.164064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a130641-45fa-4d5d-9cb1-e0aaa76e3056 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:40.203697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4a0fc-d87e-4dc1-9448-d62546cbcabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:40.204701 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4a0fc-d87e-4dc1-9448-d62546cbcabe None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:40.205197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4a0fc-d87e-4dc1-9448-d62546cbcabe None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:40.205679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4a0fc-d87e-4dc1-9448-d62546cbcabe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:40.206678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4a0fc-d87e-4dc1-9448-d62546cbcabe None admin] RBAC: Adding query filter params (role.id=b0c99aa3196841039c3bb72b951c33fb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=36f9577e047a406480fadc4b9f8c6234) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:40.207254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4a0fc-d87e-4dc1-9448-d62546cbcabe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:40.209046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4a0fc-d87e-4dc1-9448-d62546cbcabe None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:40.216117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 448/902] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:40 2022] GET /identity/v3/role_assignments?user.id=36f9577e047a406480fadc4b9f8c6234&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=b0c99aa3196841039c3bb72b951c33fb => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:40.673770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dccd8-8e16-44c9-a2b2-b73b3f7432bc None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:40.674253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dccd8-8e16-44c9-a2b2-b73b3f7432bc None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:40.674253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dccd8-8e16-44c9-a2b2-b73b3f7432bc None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:40.675597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 455/903] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:40.680908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80fa3a-900b-4c88-9d3f-c8b23ff3067c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:40.681306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80fa3a-900b-4c88-9d3f-c8b23ff3067c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:40.681485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80fa3a-900b-4c88-9d3f-c8b23ff3067c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:40.699567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4a80fa3a-900b-4c88-9d3f-c8b23ff3067c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:40.700755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4a80fa3a-900b-4c88-9d3f-c8b23ff3067c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:40.769383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 449/904] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:40.792562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dccd8-8e16-44c9-a2b2-b73b3f7432bc None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:40.900334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dccd8-8e16-44c9-a2b2-b73b3f7432bc None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:40.936183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe2632-560b-484b-a6b9-4791512b973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:40.937128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe2632-560b-484b-a6b9-4791512b973d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:40.937635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe2632-560b-484b-a6b9-4791512b973d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:40.938095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe2632-560b-484b-a6b9-4791512b973d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:40.938870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe2632-560b-484b-a6b9-4791512b973d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:40.940438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe2632-560b-484b-a6b9-4791512b973d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-1ebe2632-560b-484b-a6b9-4791512b973d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:40.951241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:40.952185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 09:51:40.953046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:40.955794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 456/905] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Aug 10 09:51:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:40.962991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80fa3a-900b-4c88-9d3f-c8b23ff3067c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:40.975296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:41.065378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80fa3a-900b-4c88-9d3f-c8b23ff3067c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:41.100477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe1cde-1e9f-4eaf-a894-46e90d4f033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:41.101136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe1cde-1e9f-4eaf-a894-46e90d4f033e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:41.101413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe1cde-1e9f-4eaf-a894-46e90d4f033e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:41.101685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe1cde-1e9f-4eaf-a894-46e90d4f033e None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:41.102315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe1cde-1e9f-4eaf-a894-46e90d4f033e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:41.102615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe1cde-1e9f-4eaf-a894-46e90d4f033e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:41.104088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe1cde-1e9f-4eaf-a894-46e90d4f033e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:41.109069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 450/906] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:41.114663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe2632-560b-484b-a6b9-4791512b973d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:41.126871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:41.186474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77c771-8ad2-4f65-89e5-61423c8b645a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:41.186769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77c771-8ad2-4f65-89e5-61423c8b645a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:41.186939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77c771-8ad2-4f65-89e5-61423c8b645a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:41.188548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 451/907] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:41.193347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2c43d-5bae-472d-a502-abec7a47c9b9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:41.193626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2c43d-5bae-472d-a502-abec7a47c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:41.193894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2c43d-5bae-472d-a502-abec7a47c9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:41.212301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6fd2c43d-5bae-472d-a502-abec7a47c9b9 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:41.213544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6fd2c43d-5bae-472d-a502-abec7a47c9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:41.229490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe2632-560b-484b-a6b9-4791512b973d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:41.273537 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e49cf-3738-4f85-bd04-aa654192b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:41.274367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e49cf-3738-4f85-bd04-aa654192b57b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:41.274652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e49cf-3738-4f85-bd04-aa654192b57b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:41.274945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e49cf-3738-4f85-bd04-aa654192b57b None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:41.276093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e49cf-3738-4f85-bd04-aa654192b57b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:41.276514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e49cf-3738-4f85-bd04-aa654192b57b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:41.278517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e49cf-3738-4f85-bd04-aa654192b57b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:41.284078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 457/909] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:41.287019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 452/909] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:41.289739 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e49cf-3738-4f85-bd04-aa654192b57b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:41.305609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2c43d-5bae-472d-a502-abec7a47c9b9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:41.314306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:41.390550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:41.422735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2c43d-5bae-472d-a502-abec7a47c9b9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:41.430469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e49cf-3738-4f85-bd04-aa654192b57b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:41.468839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433801a-db84-4025-b288-b5146cea545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:41.469355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433801a-db84-4025-b288-b5146cea545b None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:41.469638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433801a-db84-4025-b288-b5146cea545b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:41.469907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433801a-db84-4025-b288-b5146cea545b None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:41.470559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433801a-db84-4025-b288-b5146cea545b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:41.471661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2bcca-9f34-403c-95a0-2527566bc573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:41.471937 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433801a-db84-4025-b288-b5146cea545b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:41.472625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2bcca-9f34-403c-95a0-2527566bc573 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:41.473067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2bcca-9f34-403c-95a0-2527566bc573 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:41.473501 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2bcca-9f34-403c-95a0-2527566bc573 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:41.474386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2bcca-9f34-403c-95a0-2527566bc573 None admin] RBAC: Adding query filter params (interface=public, service_id=31610da65b514b63956e5a0a768c6ff4, region_id=RegionOne) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:41.474860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2bcca-9f34-403c-95a0-2527566bc573 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:41.477239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2bcca-9f34-403c-95a0-2527566bc573 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:41.484495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 458/910] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 10 09:51:41 2022] GET /identity/v3/endpoints?service_id=31610da65b514b63956e5a0a768c6ff4&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:41.488155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 453/911] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:51:41 2022] POST /identity/v3/services => generated 241 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:51:41.492942 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2bcca-9f34-403c-95a0-2527566bc573 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:41.607784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2bcca-9f34-403c-95a0-2527566bc573 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:41.648036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:41.652452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c6653-ab86-4849-8518-85aca95ae9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:41.653353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6653-ab86-4849-8518-85aca95ae9cb None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:41.653829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6653-ab86-4849-8518-85aca95ae9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:41.654275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6653-ab86-4849-8518-85aca95ae9cb None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:41.655105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c6653-ab86-4849-8518-85aca95ae9cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:41.656452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c6653-ab86-4849-8518-85aca95ae9cb None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:41.662659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 459/912] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 10 09:51:41 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:51:42.254728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333226e3-af15-426b-8f31-6501a0eec33c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.255276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333226e3-af15-426b-8f31-6501a0eec33c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.255276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333226e3-af15-426b-8f31-6501a0eec33c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.256892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 454/913] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:42.262161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c15aa-59fe-4f12-9f21-6e207f22cdc9 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.262557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c15aa-59fe-4f12-9f21-6e207f22cdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.262763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c15aa-59fe-4f12-9f21-6e207f22cdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.281290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-5f7c15aa-59fe-4f12-9f21-6e207f22cdc9 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:42.282468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5f7c15aa-59fe-4f12-9f21-6e207f22cdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:42.321973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c33d7-d0dd-4cd9-ad53-7ee4daa57da6 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.322311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c33d7-d0dd-4cd9-ad53-7ee4daa57da6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.322521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c33d7-d0dd-4cd9-ad53-7ee4daa57da6 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.324132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 455/914] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:42.328901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640486d-b718-424c-87a6-bb572f2f8058 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.329186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640486d-b718-424c-87a6-bb572f2f8058 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.329459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640486d-b718-424c-87a6-bb572f2f8058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.347486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c640486d-b718-424c-87a6-bb572f2f8058 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:42.348868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c640486d-b718-424c-87a6-bb572f2f8058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:42.354896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 460/915] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:42.376982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c15aa-59fe-4f12-9f21-6e207f22cdc9 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:42.423470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:42.438990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 456/916] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:42.458807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640486d-b718-424c-87a6-bb572f2f8058 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:42.513021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c15aa-59fe-4f12-9f21-6e207f22cdc9 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:42.548995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:42.562499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1033eb-419f-427a-a493-f1aa6def252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:42.563436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1033eb-419f-427a-a493-f1aa6def252b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.563906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1033eb-419f-427a-a493-f1aa6def252b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.564384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1033eb-419f-427a-a493-f1aa6def252b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.573383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640486d-b718-424c-87a6-bb572f2f8058 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:42.575552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1033eb-419f-427a-a493-f1aa6def252b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-3d1033eb-419f-427a-a493-f1aa6def252b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:42.580579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:42.581488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:51:42.582342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:42.590277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 461/917] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 10 09:51:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:42.591411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1033eb-419f-427a-a493-f1aa6def252b None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:42.613458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee58d6-c613-407b-86a1-8d204917cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:42.614094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee58d6-c613-407b-86a1-8d204917cd99 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.614374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee58d6-c613-407b-86a1-8d204917cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.614648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee58d6-c613-407b-86a1-8d204917cd99 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.615790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee58d6-c613-407b-86a1-8d204917cd99 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:42.617502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee58d6-c613-407b-86a1-8d204917cd99 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-baee58d6-c613-407b-86a1-8d204917cd99 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:42.630582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:42.631616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 09:51:42.632971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:42.637242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 457/918] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:51:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:42.638102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee58d6-c613-407b-86a1-8d204917cd99 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:42.703274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:42.723861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1033eb-419f-427a-a493-f1aa6def252b None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:42.754288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee58d6-c613-407b-86a1-8d204917cd99 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:42.760287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72c034-8bce-4015-941b-749d2f004513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:42.761171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72c034-8bce-4015-941b-749d2f004513 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.761703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72c034-8bce-4015-941b-749d2f004513 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.762165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72c034-8bce-4015-941b-749d2f004513 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.763103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72c034-8bce-4015-941b-749d2f004513 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:42.763610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72c034-8bce-4015-941b-749d2f004513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:42.765165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72c034-8bce-4015-941b-749d2f004513 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:42.778762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 462/919] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 10 09:51:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:42.780042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72c034-8bce-4015-941b-749d2f004513 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:42.794991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e817d-48c8-4c97-a300-2bb646559645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:42.795894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e817d-48c8-4c97-a300-2bb646559645 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.796186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e817d-48c8-4c97-a300-2bb646559645 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.796462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e817d-48c8-4c97-a300-2bb646559645 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.797101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e817d-48c8-4c97-a300-2bb646559645 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:42.797405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e817d-48c8-4c97-a300-2bb646559645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:42.798966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e817d-48c8-4c97-a300-2bb646559645 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:42.807031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 458/920] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:51:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:42.811398 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e817d-48c8-4c97-a300-2bb646559645 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:42.867348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:42.920419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e817d-48c8-4c97-a300-2bb646559645 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:42.921910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72c034-8bce-4015-941b-749d2f004513 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:42.950561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:42.966152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcb359-a456-493b-a35b-7f21dd1a8488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:42.966973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a59007-15cc-4d63-98ad-2b435b8977c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:42.967375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcb359-a456-493b-a35b-7f21dd1a8488 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.967583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a59007-15cc-4d63-98ad-2b435b8977c9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:42.967862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a59007-15cc-4d63-98ad-2b435b8977c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.968136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a59007-15cc-4d63-98ad-2b435b8977c9 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.968423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcb359-a456-493b-a35b-7f21dd1a8488 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:42.968764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a59007-15cc-4d63-98ad-2b435b8977c9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:42.969064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a59007-15cc-4d63-98ad-2b435b8977c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:42.969247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcb359-a456-493b-a35b-7f21dd1a8488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:42.970406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a59007-15cc-4d63-98ad-2b435b8977c9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:42.979165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 459/921] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 10 09:51:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:42.984404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a59007-15cc-4d63-98ad-2b435b8977c9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:42.985936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcb359-a456-493b-a35b-7f21dd1a8488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:42.987777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcb359-a456-493b-a35b-7f21dd1a8488 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-d9dcb359-a456-493b-a35b-7f21dd1a8488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:43.003571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:43.004531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:43.005392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:43.005392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:43.005392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:43.011996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 463/922] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:43.013230 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcb359-a456-493b-a35b-7f21dd1a8488 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:43.110003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a59007-15cc-4d63-98ad-2b435b8977c9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:43.119197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:43.123430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcb359-a456-493b-a35b-7f21dd1a8488 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:43.160276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e44b0-f3f4-4ea7-8ce8-d5bd4b6b09ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:43.161129 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e44b0-f3f4-4ea7-8ce8-d5bd4b6b09ba None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.161297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da444e3a-2be7-421f-a2ba-50ecb3b3cdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:43.161734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e44b0-f3f4-4ea7-8ce8-d5bd4b6b09ba None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.162082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da444e3a-2be7-421f-a2ba-50ecb3b3cdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.162297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e44b0-f3f4-4ea7-8ce8-d5bd4b6b09ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.162533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da444e3a-2be7-421f-a2ba-50ecb3b3cdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.162894 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da444e3a-2be7-421f-a2ba-50ecb3b3cdc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.163996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e44b0-f3f4-4ea7-8ce8-d5bd4b6b09ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:43.164338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da444e3a-2be7-421f-a2ba-50ecb3b3cdc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:43.164731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da444e3a-2be7-421f-a2ba-50ecb3b3cdc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:43.165801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e44b0-f3f4-4ea7-8ce8-d5bd4b6b09ba None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:43.166256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da444e3a-2be7-421f-a2ba-50ecb3b3cdc3 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:43.184366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 464/923] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:43.185571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da444e3a-2be7-421f-a2ba-50ecb3b3cdc3 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:43.203536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 460/924] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:51:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:43.304265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da444e3a-2be7-421f-a2ba-50ecb3b3cdc3 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:43.353937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1360f-1283-418b-911b-a478eba9d267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:43.354979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1360f-1283-418b-911b-a478eba9d267 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.355460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1360f-1283-418b-911b-a478eba9d267 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.355949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1360f-1283-418b-911b-a478eba9d267 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.364840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1360f-1283-418b-911b-a478eba9d267 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:43.366729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1360f-1283-418b-911b-a478eba9d267 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-32b1360f-1283-418b-911b-a478eba9d267 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:43.379864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:43.380844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:43.382306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:43.384831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 465/925] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:51:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:43.390913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e44b0-f3f4-4ea7-8ce8-d5bd4b6b09ba None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:43.438542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:43.501426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e44b0-f3f4-4ea7-8ce8-d5bd4b6b09ba None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:43.539373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6d0fc-eb66-4791-9d8c-39657a7d4a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:43.540350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d0fc-eb66-4791-9d8c-39657a7d4a07 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.540828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d0fc-eb66-4791-9d8c-39657a7d4a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.541324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d0fc-eb66-4791-9d8c-39657a7d4a07 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.542251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d0fc-eb66-4791-9d8c-39657a7d4a07 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:43.542752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d0fc-eb66-4791-9d8c-39657a7d4a07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:43.544385 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d0fc-eb66-4791-9d8c-39657a7d4a07 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:43.563365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 461/926] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 10 09:51:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:43.574995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1360f-1283-418b-911b-a478eba9d267 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:43.579489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9ea11-27c3-4434-801e-509ac562fc3d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.579773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9ea11-27c3-4434-801e-509ac562fc3d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.580045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9ea11-27c3-4434-801e-509ac562fc3d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.585129 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 462/927] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:43.586513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeb848-beec-46da-a816-fd6f32a119c5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.586795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeb848-beec-46da-a816-fd6f32a119c5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.587114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeb848-beec-46da-a816-fd6f32a119c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.589464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:43.606610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-83aeb848-beec-46da-a816-fd6f32a119c5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:43.607918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-83aeb848-beec-46da-a816-fd6f32a119c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:43.676154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1360f-1283-418b-911b-a478eba9d267 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:43.685977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 463/928] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:43.702922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeb848-beec-46da-a816-fd6f32a119c5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:43.712544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0b8e2-289a-447b-bca3-53b6b39cacce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:43.713468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0b8e2-289a-447b-bca3-53b6b39cacce None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.713974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0b8e2-289a-447b-bca3-53b6b39cacce None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.714441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0b8e2-289a-447b-bca3-53b6b39cacce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.726272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0b8e2-289a-447b-bca3-53b6b39cacce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:43.727831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0b8e2-289a-447b-bca3-53b6b39cacce None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:43.733548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-3ef0b8e2-289a-447b-bca3-53b6b39cacce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:43.742224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:43.743200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:43.744355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:43.744355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:43.744355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:43.748151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 466/929] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:43.749351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705611a0-bb0e-48a3-9f28-2cd7ca9f1dc3 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.749746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705611a0-bb0e-48a3-9f28-2cd7ca9f1dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.749910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705611a0-bb0e-48a3-9f28-2cd7ca9f1dc3 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.751177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 467/930] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:43.751866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705611a0-bb0e-48a3-9f28-2cd7ca9f1dc3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:43.814547 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeb848-beec-46da-a816-fd6f32a119c5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:43.823233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:43.850344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520bb2d-4163-451a-8055-bfa735f79492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:43.851926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520bb2d-4163-451a-8055-bfa735f79492 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.852393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520bb2d-4163-451a-8055-bfa735f79492 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.852736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520bb2d-4163-451a-8055-bfa735f79492 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.853382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520bb2d-4163-451a-8055-bfa735f79492 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:43.854790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520bb2d-4163-451a-8055-bfa735f79492 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:43.861182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705611a0-bb0e-48a3-9f28-2cd7ca9f1dc3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-8520bb2d-4163-451a-8055-bfa735f79492 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:43.864006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:43.865146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:43.865959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 464/931] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:43.866882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ee40d-c879-4536-8fa5-af4284b1c1f3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.866882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ee40d-c879-4536-8fa5-af4284b1c1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.866882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ee40d-c879-4536-8fa5-af4284b1c1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.885386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-124ee40d-c879-4536-8fa5-af4284b1c1f3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:43.886542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-124ee40d-c879-4536-8fa5-af4284b1c1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:43.904789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0944e40-4ec6-45a5-b267-2ee441828d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:43.905523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944e40-4ec6-45a5-b267-2ee441828d4c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:43.905899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944e40-4ec6-45a5-b267-2ee441828d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:43.906270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944e40-4ec6-45a5-b267-2ee441828d4c None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:43.907036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0944e40-4ec6-45a5-b267-2ee441828d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:43.907435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0944e40-4ec6-45a5-b267-2ee441828d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:43.908895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0944e40-4ec6-45a5-b267-2ee441828d4c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:43.921724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 468/932] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:43.926866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0944e40-4ec6-45a5-b267-2ee441828d4c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:43.956181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 465/933] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:43.957208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ee40d-c879-4536-8fa5-af4284b1c1f3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:43.991812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:44.035384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0944e40-4ec6-45a5-b267-2ee441828d4c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.063795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:44.067586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ee40d-c879-4536-8fa5-af4284b1c1f3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.074295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858339-3335-4944-b790-fd06f869aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.074994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858339-3335-4944-b790-fd06f869aac7 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.075336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858339-3335-4944-b790-fd06f869aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.075672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858339-3335-4944-b790-fd06f869aac7 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.076373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858339-3335-4944-b790-fd06f869aac7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:44.076731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858339-3335-4944-b790-fd06f869aac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.078234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858339-3335-4944-b790-fd06f869aac7 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.088156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 469/934] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:44.088156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858339-3335-4944-b790-fd06f869aac7 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:44.109208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047d192-e8bc-4d2b-a2e2-707d02c6a156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.109860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047d192-e8bc-4d2b-a2e2-707d02c6a156 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.110201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047d192-e8bc-4d2b-a2e2-707d02c6a156 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.110531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047d192-e8bc-4d2b-a2e2-707d02c6a156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.117385 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047d192-e8bc-4d2b-a2e2-707d02c6a156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.119134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047d192-e8bc-4d2b-a2e2-707d02c6a156 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c047d192-e8bc-4d2b-a2e2-707d02c6a156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:44.130439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:44.132432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:44.133554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:44.133554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:44.133554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:44.133554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:44.133554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:44.133554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:44.133554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:44.133554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:44.137793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 466/935] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:44.138188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047d192-e8bc-4d2b-a2e2-707d02c6a156 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:44.193770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858339-3335-4944-b790-fd06f869aac7 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.223615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:44.230405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b3868-e76c-47e6-a46f-d899e91bb0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.231055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b3868-e76c-47e6-a46f-d899e91bb0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.231328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b3868-e76c-47e6-a46f-d899e91bb0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.231595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b3868-e76c-47e6-a46f-d899e91bb0d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.232168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b3868-e76c-47e6-a46f-d899e91bb0d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.233507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b3868-e76c-47e6-a46f-d899e91bb0d0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.240655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047d192-e8bc-4d2b-a2e2-707d02c6a156 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-051b3868-e76c-47e6-a46f-d899e91bb0d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:44.244176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:44.245287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:44.246393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 470/936] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Aug 10 09:51:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:44.247671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b3868-e76c-47e6-a46f-d899e91bb0d0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:44.281084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca6049-d992-4ca3-bb11-3860168b731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.281694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca6049-d992-4ca3-bb11-3860168b731c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.281967 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca6049-d992-4ca3-bb11-3860168b731c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.282234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca6049-d992-4ca3-bb11-3860168b731c None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.282853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca6049-d992-4ca3-bb11-3860168b731c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:44.283174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca6049-d992-4ca3-bb11-3860168b731c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.284513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca6049-d992-4ca3-bb11-3860168b731c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.290261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 467/937] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:44.290944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca6049-d992-4ca3-bb11-3860168b731c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:44.314060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:44.365994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b3868-e76c-47e6-a46f-d899e91bb0d0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.396409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca6049-d992-4ca3-bb11-3860168b731c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.407198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065e11-d070-419b-9e95-6fe1955a7b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.407915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065e11-d070-419b-9e95-6fe1955a7b97 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.410625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065e11-d070-419b-9e95-6fe1955a7b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.411589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065e11-d070-419b-9e95-6fe1955a7b97 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.413003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065e11-d070-419b-9e95-6fe1955a7b97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:44.413194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065e11-d070-419b-9e95-6fe1955a7b97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.415100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065e11-d070-419b-9e95-6fe1955a7b97 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.428138 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 471/938] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:44.429461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065e11-d070-419b-9e95-6fe1955a7b97 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:44.437407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c58c65-6616-4ed2-b132-b76dcc42a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.438041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c58c65-6616-4ed2-b132-b76dcc42a359 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.438324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c58c65-6616-4ed2-b132-b76dcc42a359 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.438598 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c58c65-6616-4ed2-b132-b76dcc42a359 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.439286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c58c65-6616-4ed2-b132-b76dcc42a359 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:44.439592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c58c65-6616-4ed2-b132-b76dcc42a359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.440976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c58c65-6616-4ed2-b132-b76dcc42a359 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.451274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 468/939] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:44.451274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c58c65-6616-4ed2-b132-b76dcc42a359 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:44.463965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:44.539912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065e11-d070-419b-9e95-6fe1955a7b97 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.551771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c58c65-6616-4ed2-b132-b76dcc42a359 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.557881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:44.579310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6da85f-7535-4a40-8fb6-3ef402ca11ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.579907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6da85f-7535-4a40-8fb6-3ef402ca11ce None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.580178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6da85f-7535-4a40-8fb6-3ef402ca11ce None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.580442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6da85f-7535-4a40-8fb6-3ef402ca11ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.581126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6da85f-7535-4a40-8fb6-3ef402ca11ce None admin] RBAC: Adding query filter params (interface=public, service_id=9f6e39e8c60c4420a563498a94536855, region_id=RegionOne) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:44.581426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6da85f-7535-4a40-8fb6-3ef402ca11ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.582773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6da85f-7535-4a40-8fb6-3ef402ca11ce None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.588452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 472/940] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 10 09:51:44 2022] GET /identity/v3/endpoints?service_id=9f6e39e8c60c4420a563498a94536855&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:44.589209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6da85f-7535-4a40-8fb6-3ef402ca11ce None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:44.595852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96e4e2-e223-4d08-8243-1243702c58f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.596512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96e4e2-e223-4d08-8243-1243702c58f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.596788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96e4e2-e223-4d08-8243-1243702c58f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.597059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96e4e2-e223-4d08-8243-1243702c58f2 None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/36f9577e047a406480fadc4b9f8c6234/roles/b0c99aa3196841039c3bb72b951c33fb` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.599202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96e4e2-e223-4d08-8243-1243702c58f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=36f9577e047a406480fadc4b9f8c6234, role_id=b0c99aa3196841039c3bb72b951c33fb)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.600613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96e4e2-e223-4d08-8243-1243702c58f2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.614144 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 469/941] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:51:44 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/36f9577e047a406480fadc4b9f8c6234/roles/b0c99aa3196841039c3bb72b951c33fb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:51:44.615224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96e4e2-e223-4d08-8243-1243702c58f2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:44.716983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6da85f-7535-4a40-8fb6-3ef402ca11ce None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.717167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:44.762996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00500068-98f2-4379-af87-f82221f1db8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.763633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00500068-98f2-4379-af87-f82221f1db8a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.763903 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00500068-98f2-4379-af87-f82221f1db8a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.764168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00500068-98f2-4379-af87-f82221f1db8a None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.764882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00500068-98f2-4379-af87-f82221f1db8a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:44.765181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00500068-98f2-4379-af87-f82221f1db8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.766317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96e4e2-e223-4d08-8243-1243702c58f2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.766611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00500068-98f2-4379-af87-f82221f1db8a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.774998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 473/942] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:44.778507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00500068-98f2-4379-af87-f82221f1db8a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:44.816698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c4d1a-f89e-46db-8ea2-3011a4f81378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.817323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4d1a-f89e-46db-8ea2-3011a4f81378 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.817604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4d1a-f89e-46db-8ea2-3011a4f81378 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.817877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4d1a-f89e-46db-8ea2-3011a4f81378 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.818475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c4d1a-f89e-46db-8ea2-3011a4f81378 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.819938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c4d1a-f89e-46db-8ea2-3011a4f81378 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.827841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 470/943] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 10 09:51:44 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:51:44.832452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:44.900745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00500068-98f2-4379-af87-f82221f1db8a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:44.940034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbbd08-0193-4e3e-8d94-dd9a3559672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:44.940660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbbd08-0193-4e3e-8d94-dd9a3559672a None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:44.940941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbbd08-0193-4e3e-8d94-dd9a3559672a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:44.941208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbbd08-0193-4e3e-8d94-dd9a3559672a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:44.941878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbbd08-0193-4e3e-8d94-dd9a3559672a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:44.943303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbbd08-0193-4e3e-8d94-dd9a3559672a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:44.969601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 474/944] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:51:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:45.510204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7844cf-3066-435f-a9ef-bed67aec3209 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:45.510767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7844cf-3066-435f-a9ef-bed67aec3209 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:45.510767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7844cf-3066-435f-a9ef-bed67aec3209 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:45.512364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 471/945] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:45.528621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713042a2-27a4-46cb-a794-bba7e6b9340f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:45.529103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713042a2-27a4-46cb-a794-bba7e6b9340f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:45.529198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713042a2-27a4-46cb-a794-bba7e6b9340f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:45.553320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-713042a2-27a4-46cb-a794-bba7e6b9340f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:45.554552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-713042a2-27a4-46cb-a794-bba7e6b9340f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:45.680412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 475/946] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:45.703433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7844cf-3066-435f-a9ef-bed67aec3209 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:45.807515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7844cf-3066-435f-a9ef-bed67aec3209 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:45.842030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:45.842633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:45.842977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:45.843259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:45.843957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:45.845368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:45.848043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WARNING keystone.server.flask.common [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 09:51:45.852468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:45.869881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.sql.core [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 09:51:45.869881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 09:51:45.869881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [parameters: {'id': '2500dc43ac1f414b8bb53ed1f72bbcac', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 09:51:45.869881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80247) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2500dc43ac1f414b8bb53ed1f72bbcac.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2500dc43ac1f414b8bb53ed1f72bbcac. Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application conn.query(q) Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 09:51:45.877845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result.read() Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 09:51:45.878730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application next(self.gen) Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application yield resource Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application next(self.gen) Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise self.value Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application session.commit() Aug 10 09:51:45.879711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.session.flush() Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application compat.raise_( Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 09:51:45.880587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 09:51:45.881467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application conn.query(q) Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result.read() Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 09:51:45.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [parameters: {'id': '2500dc43ac1f414b8bb53ed1f72bbcac', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:45.883715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:45.886477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2500dc43ac1f414b8bb53ed1f72bbcac. Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:45.888725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 472/947] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Aug 10 09:51:45 2022] POST /identity/v3/projects => generated 262 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:45.895820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713042a2-27a4-46cb-a794-bba7e6b9340f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:45.916462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:45.991685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713042a2-27a4-46cb-a794-bba7e6b9340f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:46.026466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df0bf6-dc0e-4f1b-8d97-ca132b08ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:46.027179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df0bf6-dc0e-4f1b-8d97-ca132b08ffda None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:46.027454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df0bf6-dc0e-4f1b-8d97-ca132b08ffda None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:46.027724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df0bf6-dc0e-4f1b-8d97-ca132b08ffda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:46.034591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df0bf6-dc0e-4f1b-8d97-ca132b08ffda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:46.036051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df0bf6-dc0e-4f1b-8d97-ca132b08ffda None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-25df0bf6-dc0e-4f1b-8d97-ca132b08ffda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:46.047919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:46.049861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:46.052017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:46.052017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:46.052017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:46.052017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:46.052017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:46.052017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:46.052017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:46.052017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:46.052017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 476/948] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:46.055347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:46.160787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffb723-d702-4a4a-955e-fda1ebfc7a49 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:46.196755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df3d21-03d7-4879-878e-5684d7bea7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:46.197369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df3d21-03d7-4879-878e-5684d7bea7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:46.197650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df3d21-03d7-4879-878e-5684d7bea7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:46.197919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df3d21-03d7-4879-878e-5684d7bea7a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:46.198578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df3d21-03d7-4879-878e-5684d7bea7a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:46.198878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df3d21-03d7-4879-878e-5684d7bea7a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:46.200534 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df3d21-03d7-4879-878e-5684d7bea7a6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:46.221896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 473/949] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:46 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:46.789298 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b30fe-fc92-42cb-adc6-5af60400d1b7 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:46.789802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b30fe-fc92-42cb-adc6-5af60400d1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:46.789802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b30fe-fc92-42cb-adc6-5af60400d1b7 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:46.791400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 477/950] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:46.796662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f70b84-3a81-4a66-af08-2f32693608f7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:46.797087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f70b84-3a81-4a66-af08-2f32693608f7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:46.797279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f70b84-3a81-4a66-af08-2f32693608f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:46.817666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-48f70b84-3a81-4a66-af08-2f32693608f7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:46.820159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-48f70b84-3a81-4a66-af08-2f32693608f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:46.867256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:46.895108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 474/951] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:46.912073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b30fe-fc92-42cb-adc6-5af60400d1b7 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.005758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724de850-e32a-4456-a101-b61e657d7cfd None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.007316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724de850-e32a-4456-a101-b61e657d7cfd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.008135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724de850-e32a-4456-a101-b61e657d7cfd None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.010408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 475/952] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:47.010546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:47.015726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86201989-0c67-46af-94ee-82cdd5e7e1fa None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.016260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86201989-0c67-46af-94ee-82cdd5e7e1fa None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.016756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86201989-0c67-46af-94ee-82cdd5e7e1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.027311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b30fe-fc92-42cb-adc6-5af60400d1b7 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.036039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-86201989-0c67-46af-94ee-82cdd5e7e1fa None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:47.037481 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-86201989-0c67-46af-94ee-82cdd5e7e1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:47.072016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed110c-a252-447c-be76-2241477bb6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.072651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed110c-a252-447c-be76-2241477bb6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.072943 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed110c-a252-447c-be76-2241477bb6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.073201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed110c-a252-447c-be76-2241477bb6f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.086026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed110c-a252-447c-be76-2241477bb6f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-4aed110c-a252-447c-be76-2241477bb6f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:47.091421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:47.092399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:47.098456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 478/953] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 10 09:51:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:47.100873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed110c-a252-447c-be76-2241477bb6f0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.139064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 476/954] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:47.155459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86201989-0c67-46af-94ee-82cdd5e7e1fa None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.214302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed110c-a252-447c-be76-2241477bb6f0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.249837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d17577-44ef-4b13-93cb-7cc800a41438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.250439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d17577-44ef-4b13-93cb-7cc800a41438 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.250718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d17577-44ef-4b13-93cb-7cc800a41438 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.251016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d17577-44ef-4b13-93cb-7cc800a41438 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.251677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d17577-44ef-4b13-93cb-7cc800a41438 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:47.251981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d17577-44ef-4b13-93cb-7cc800a41438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:47.253364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d17577-44ef-4b13-93cb-7cc800a41438 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:47.262519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86201989-0c67-46af-94ee-82cdd5e7e1fa None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.267436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 479/955] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:47.267436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e017e1-279b-4e4b-8d6f-24170fdbceaa None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.267436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e017e1-279b-4e4b-8d6f-24170fdbceaa None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.267436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e017e1-279b-4e4b-8d6f-24170fdbceaa None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.267436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 480/956] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:47.270970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e017e1-279b-4e4b-8d6f-24170fdbceaa None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.281938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:47.284744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:47.306299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54508e35-b37a-4559-a3b6-1eefb720fdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.307046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54508e35-b37a-4559-a3b6-1eefb720fdaf None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.307326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54508e35-b37a-4559-a3b6-1eefb720fdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.307595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54508e35-b37a-4559-a3b6-1eefb720fdaf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.308175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54508e35-b37a-4559-a3b6-1eefb720fdaf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:47.309519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54508e35-b37a-4559-a3b6-1eefb720fdaf None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-54508e35-b37a-4559-a3b6-1eefb720fdaf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:47.318112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:47.319817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:47.320777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 477/957] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:47.321599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7b644-130a-4836-b285-631ec85ddfd5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.321673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7b644-130a-4836-b285-631ec85ddfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.321890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7b644-130a-4836-b285-631ec85ddfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.339792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-76f7b644-130a-4836-b285-631ec85ddfd5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:47.340880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-76f7b644-130a-4836-b285-631ec85ddfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:47.385019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e017e1-279b-4e4b-8d6f-24170fdbceaa None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.418304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 478/958] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:47.419169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7b644-130a-4836-b285-631ec85ddfd5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.446888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc450778-64dd-4aac-8cf7-5e37c22546f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.448098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc450778-64dd-4aac-8cf7-5e37c22546f7 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.448557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc450778-64dd-4aac-8cf7-5e37c22546f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.449126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc450778-64dd-4aac-8cf7-5e37c22546f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.464251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc450778-64dd-4aac-8cf7-5e37c22546f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:47.466704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc450778-64dd-4aac-8cf7-5e37c22546f7 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-fc450778-64dd-4aac-8cf7-5e37c22546f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:47.483675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:47.484659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:47.485622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:47.485622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:47.485622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:47.485622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 481/959] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:47.485622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc450778-64dd-4aac-8cf7-5e37c22546f7 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.524539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7b644-130a-4836-b285-631ec85ddfd5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.534143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:47.550184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:47.563021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8884272-2541-4498-8673-89a1263fc424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.563628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8884272-2541-4498-8673-89a1263fc424 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.563906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8884272-2541-4498-8673-89a1263fc424 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.564173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8884272-2541-4498-8673-89a1263fc424 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.564799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8884272-2541-4498-8673-89a1263fc424 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:47.565099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8884272-2541-4498-8673-89a1263fc424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:47.566451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8884272-2541-4498-8673-89a1263fc424 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:47.571344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 479/960] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:47.572118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8884272-2541-4498-8673-89a1263fc424 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.601740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc450778-64dd-4aac-8cf7-5e37c22546f7 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.636207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff31bcf-8431-423d-bb16-6f07f7c6001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.636859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31bcf-8431-423d-bb16-6f07f7c6001d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.637137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31bcf-8431-423d-bb16-6f07f7c6001d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.637411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31bcf-8431-423d-bb16-6f07f7c6001d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.637975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff31bcf-8431-423d-bb16-6f07f7c6001d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:47.639371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff31bcf-8431-423d-bb16-6f07f7c6001d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-3ff31bcf-8431-423d-bb16-6f07f7c6001d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:47.648406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:47.649405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:47.650245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 482/961] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:47.651074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff31bcf-8431-423d-bb16-6f07f7c6001d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.684819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8884272-2541-4498-8673-89a1263fc424 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.718117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad428b-c9cc-450b-b0e5-1d27ab43598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.718724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad428b-c9cc-450b-b0e5-1d27ab43598f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.719024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad428b-c9cc-450b-b0e5-1d27ab43598f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.719298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad428b-c9cc-450b-b0e5-1d27ab43598f None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.719944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad428b-c9cc-450b-b0e5-1d27ab43598f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:47.720247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad428b-c9cc-450b-b0e5-1d27ab43598f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:47.721597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad428b-c9cc-450b-b0e5-1d27ab43598f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:47.738143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 480/962] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:47.738143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad428b-c9cc-450b-b0e5-1d27ab43598f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.759095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff31bcf-8431-423d-bb16-6f07f7c6001d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.766928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:47.784800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:47.797038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6996d0-5766-4b59-8f18-ba91daa9f80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.797702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6996d0-5766-4b59-8f18-ba91daa9f80a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.797963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6996d0-5766-4b59-8f18-ba91daa9f80a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.798260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6996d0-5766-4b59-8f18-ba91daa9f80a None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.798885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6996d0-5766-4b59-8f18-ba91daa9f80a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:47.799241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6996d0-5766-4b59-8f18-ba91daa9f80a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:47.800587 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6996d0-5766-4b59-8f18-ba91daa9f80a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:47.805517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 483/963] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:47.806122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6996d0-5766-4b59-8f18-ba91daa9f80a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.845692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad428b-c9cc-450b-b0e5-1d27ab43598f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.881241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac404f15-dace-42b8-a078-75734f3b80b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.882287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac404f15-dace-42b8-a078-75734f3b80b8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.882566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac404f15-dace-42b8-a078-75734f3b80b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.882836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac404f15-dace-42b8-a078-75734f3b80b8 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.883489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac404f15-dace-42b8-a078-75734f3b80b8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:47.883786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac404f15-dace-42b8-a078-75734f3b80b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:47.885166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac404f15-dace-42b8-a078-75734f3b80b8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:47.889934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 481/964] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:47.892140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac404f15-dace-42b8-a078-75734f3b80b8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.914572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6996d0-5766-4b59-8f18-ba91daa9f80a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:47.948567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91c586-4624-449d-853c-2f511d0828b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:47.949313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91c586-4624-449d-853c-2f511d0828b2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:47.949653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91c586-4624-449d-853c-2f511d0828b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:47.950029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91c586-4624-449d-853c-2f511d0828b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:47.959444 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91c586-4624-449d-853c-2f511d0828b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:47.962202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91c586-4624-449d-853c-2f511d0828b2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-dc91c586-4624-449d-853c-2f511d0828b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:47.973400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:47.974389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 484/965] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:47.975381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91c586-4624-449d-853c-2f511d0828b2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:47.998740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:48.006251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac404f15-dace-42b8-a078-75734f3b80b8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:48.029377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:48.040420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48ed3f-e061-4392-9e8b-1de5a1114258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:48.041024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48ed3f-e061-4392-9e8b-1de5a1114258 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.041300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48ed3f-e061-4392-9e8b-1de5a1114258 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.041575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48ed3f-e061-4392-9e8b-1de5a1114258 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.042229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48ed3f-e061-4392-9e8b-1de5a1114258 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:48.043611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48ed3f-e061-4392-9e8b-1de5a1114258 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:48.066151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 482/966] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:51:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:48.066887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48ed3f-e061-4392-9e8b-1de5a1114258 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:48.106623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91c586-4624-449d-853c-2f511d0828b2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:48.159029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572fc1f-293c-471c-8249-7a9ea77e6f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:48.160064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572fc1f-293c-471c-8249-7a9ea77e6f0a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.160542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572fc1f-293c-471c-8249-7a9ea77e6f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.161010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572fc1f-293c-471c-8249-7a9ea77e6f0a None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.161980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572fc1f-293c-471c-8249-7a9ea77e6f0a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:48.162661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572fc1f-293c-471c-8249-7a9ea77e6f0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:48.164399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572fc1f-293c-471c-8249-7a9ea77e6f0a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:48.170511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 485/967] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:48.190152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48ed3f-e061-4392-9e8b-1de5a1114258 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:48.232613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ac039-4c89-4222-821d-f6174795e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:48.233244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ac039-4c89-4222-821d-f6174795e66d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.233521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ac039-4c89-4222-821d-f6174795e66d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.233789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ac039-4c89-4222-821d-f6174795e66d None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.234577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ac039-4c89-4222-821d-f6174795e66d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:48.234882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ac039-4c89-4222-821d-f6174795e66d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:48.236343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ac039-4c89-4222-821d-f6174795e66d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:48.258118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 483/968] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 10 09:51:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:48.263938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572fc1f-293c-471c-8249-7a9ea77e6f0a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:48.315812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95660a1c-7fb0-4f29-bf8f-e47f5f896fff None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.316384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95660a1c-7fb0-4f29-bf8f-e47f5f896fff None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.316866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95660a1c-7fb0-4f29-bf8f-e47f5f896fff None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.318413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 484/969] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:48.324890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709085-4d37-4e5f-b18e-a9711ce516a5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.325444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709085-4d37-4e5f-b18e-a9711ce516a5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.325932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709085-4d37-4e5f-b18e-a9711ce516a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.344502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-af709085-4d37-4e5f-b18e-a9711ce516a5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:48.346010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-af709085-4d37-4e5f-b18e-a9711ce516a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:48.348297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:48.390781 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572fc1f-293c-471c-8249-7a9ea77e6f0a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:48.402418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:48.431040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 485/970] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:48.434321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43852d-8fa4-413a-8946-b7495933e2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:48.435259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43852d-8fa4-413a-8946-b7495933e2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.435740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43852d-8fa4-413a-8946-b7495933e2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.436210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43852d-8fa4-413a-8946-b7495933e2c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.455564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af709085-4d37-4e5f-b18e-a9711ce516a5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:48.518402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43852d-8fa4-413a-8946-b7495933e2c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:48.520271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43852d-8fa4-413a-8946-b7495933e2c3 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:48.569781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af709085-4d37-4e5f-b18e-a9711ce516a5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:48.605353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:48.605985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.606261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.606535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.607203 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:48.610002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:48.621773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:48.625318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.sql.core [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 09:51:48.625318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 09:51:48.625318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [parameters: {'user_id': 'cca02954f7e14b788e73200a0ae8dabf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 09:51:48.625318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80247) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 09:51:48.631827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 486/971] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:48 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application conn.query(q) Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result.read() Aug 10 09:51:48.634133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 09:51:48.635459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application next(self.gen) Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application yield resource Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application next(self.gen) Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise self.value Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application session.commit() Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 09:51:48.636492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.session.flush() Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application compat.raise_( Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 09:51:48.637417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 09:51:48.638296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application conn.query(q) Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result.read() Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 09:51:48.639194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cca02954f7e14b788e73200a0ae8dabf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:48.640515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 09:51:48.642173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 09:51:48.644206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 09:51:48.644206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 09:51:48.644206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 09:51:48.644206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 09:51:48.644206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 09:51:48.644206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:48.647065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43852d-8fa4-413a-8946-b7495933e2c3 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:48.648313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 486/972] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 10 09:51:48 2022] POST /identity/v3/users => generated 160 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:48.649730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:48.659519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:48.751883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43852d-8fa4-413a-8946-b7495933e2c3 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:48.762684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46da35-87c9-4632-bfcb-8dc4044d11c4 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:48.787484 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6775c-99d2-4c0f-8e3b-50cc430281d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:48.789030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6775c-99d2-4c0f-8e3b-50cc430281d8 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.789511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6775c-99d2-4c0f-8e3b-50cc430281d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.789992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6775c-99d2-4c0f-8e3b-50cc430281d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.801233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bedaa-63da-4f92-8580-ad015f90c756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:48.801623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6775c-99d2-4c0f-8e3b-50cc430281d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:48.802068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bedaa-63da-4f92-8580-ad015f90c756 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.802258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bedaa-63da-4f92-8580-ad015f90c756 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.802529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bedaa-63da-4f92-8580-ad015f90c756 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.804930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6775c-99d2-4c0f-8e3b-50cc430281d8 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:48.810803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bedaa-63da-4f92-8580-ad015f90c756 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:48.812531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bedaa-63da-4f92-8580-ad015f90c756 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-26c6775c-99d2-4c0f-8e3b-50cc430281d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:48.820461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:48.821372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:48.822259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:48.822259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:48.822259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:48.824362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 487/973] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-1a1bedaa-63da-4f92-8580-ad015f90c756 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:48.826336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:48.827241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 09:51:48.828179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:48.831028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 487/974] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:51:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:48.833081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6775c-99d2-4c0f-8e3b-50cc430281d8 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:48.834292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bedaa-63da-4f92-8580-ad015f90c756 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:48.904526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:48.908911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:48.954458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bedaa-63da-4f92-8580-ad015f90c756 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:48.961453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6775c-99d2-4c0f-8e3b-50cc430281d8 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:48.990496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226ae3d-0d8c-4bb1-a310-a806faa2b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:48.991302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226ae3d-0d8c-4bb1-a310-a806faa2b489 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:48.991653 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226ae3d-0d8c-4bb1-a310-a806faa2b489 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:48.992022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226ae3d-0d8c-4bb1-a310-a806faa2b489 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:48.992755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226ae3d-0d8c-4bb1-a310-a806faa2b489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:48.993041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226ae3d-0d8c-4bb1-a310-a806faa2b489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:48.994839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226ae3d-0d8c-4bb1-a310-a806faa2b489 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:49.007259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0714bd-e744-4c40-9c0e-c723081d14f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:49.008084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0714bd-e744-4c40-9c0e-c723081d14f1 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:49.008557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0714bd-e744-4c40-9c0e-c723081d14f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:49.009009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0714bd-e744-4c40-9c0e-c723081d14f1 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:49.009890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0714bd-e744-4c40-9c0e-c723081d14f1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:49.010384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0714bd-e744-4c40-9c0e-c723081d14f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:49.010840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 488/975] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:49.012250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0714bd-e744-4c40-9c0e-c723081d14f1 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:49.018177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226ae3d-0d8c-4bb1-a310-a806faa2b489 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:49.051038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 488/976] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 10 09:51:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:49.140845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226ae3d-0d8c-4bb1-a310-a806faa2b489 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:49.166138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:49.185360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35c6fb-d6f5-4f62-a0fe-9330a4bd8151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:49.186876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35c6fb-d6f5-4f62-a0fe-9330a4bd8151 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:49.187431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35c6fb-d6f5-4f62-a0fe-9330a4bd8151 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:49.187910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35c6fb-d6f5-4f62-a0fe-9330a4bd8151 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:49.196597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35c6fb-d6f5-4f62-a0fe-9330a4bd8151 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:49.198455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35c6fb-d6f5-4f62-a0fe-9330a4bd8151 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9c35c6fb-d6f5-4f62-a0fe-9330a4bd8151 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:51:49.211807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:49.212790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:49.213728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:51:49.213728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:51:49.213728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:51:49.213728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:51:49.213728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:51:49.213728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:51:49.213728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:51:49.213728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:49.216335 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 489/977] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:49.223439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0714bd-e744-4c40-9c0e-c723081d14f1 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:49.333765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0714bd-e744-4c40-9c0e-c723081d14f1 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:49.352912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:49.371227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf93fb-81be-4294-9f3e-8a5783dccd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:49.372215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf93fb-81be-4294-9f3e-8a5783dccd6f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:49.372679 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf93fb-81be-4294-9f3e-8a5783dccd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:49.373136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf93fb-81be-4294-9f3e-8a5783dccd6f None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:49.374044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf93fb-81be-4294-9f3e-8a5783dccd6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:49.374533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf93fb-81be-4294-9f3e-8a5783dccd6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:49.376219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf93fb-81be-4294-9f3e-8a5783dccd6f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:49.389622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 489/978] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:51:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:49.395389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35c6fb-d6f5-4f62-a0fe-9330a4bd8151 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:49.499517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35c6fb-d6f5-4f62-a0fe-9330a4bd8151 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:49.539283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfdf4f-9aed-4c20-90db-8d0277272321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:49.540217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfdf4f-9aed-4c20-90db-8d0277272321 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:49.540647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfdf4f-9aed-4c20-90db-8d0277272321 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:49.541087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfdf4f-9aed-4c20-90db-8d0277272321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:49.541961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfdf4f-9aed-4c20-90db-8d0277272321 None admin] RBAC: Adding query filter params (role.id=b0c99aa3196841039c3bb72b951c33fb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=36f9577e047a406480fadc4b9f8c6234) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:49.542420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfdf4f-9aed-4c20-90db-8d0277272321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:49.544018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfdf4f-9aed-4c20-90db-8d0277272321 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:49.549920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 490/979] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:51:49 2022] GET /identity/v3/role_assignments?user.id=36f9577e047a406480fadc4b9f8c6234&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=b0c99aa3196841039c3bb72b951c33fb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:50.175921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d549c3-96b7-427b-88b4-e0dc8fd1c390 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:50.176491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d549c3-96b7-427b-88b4-e0dc8fd1c390 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:50.176491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d549c3-96b7-427b-88b4-e0dc8fd1c390 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:50.178001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 490/980] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:50.183530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f646c-04cb-4e75-be07-9485350523d0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:50.183987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f646c-04cb-4e75-be07-9485350523d0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:50.184064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f646c-04cb-4e75-be07-9485350523d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:50.202794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-af0f646c-04cb-4e75-be07-9485350523d0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:50.204303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-af0f646c-04cb-4e75-be07-9485350523d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:50.248208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:50.276833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 491/981] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:50.298867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d549c3-96b7-427b-88b4-e0dc8fd1c390 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:50.316248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97160e52-7ccf-456f-95e7-35e918c1faa4 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:50.316535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97160e52-7ccf-456f-95e7-35e918c1faa4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:50.316806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97160e52-7ccf-456f-95e7-35e918c1faa4 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:50.318278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 492/982] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:50.323678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad27423-2775-4f77-a4af-da8e29fda6bf None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:50.323955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad27423-2775-4f77-a4af-da8e29fda6bf None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:50.324225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad27423-2775-4f77-a4af-da8e29fda6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:50.342918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8ad27423-2775-4f77-a4af-da8e29fda6bf None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:50.344174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8ad27423-2775-4f77-a4af-da8e29fda6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:50.406544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d549c3-96b7-427b-88b4-e0dc8fd1c390 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:50.424473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 493/983] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:50.441405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e1864-37e7-42c5-a606-ebc9e517d4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:50.441971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e1864-37e7-42c5-a606-ebc9e517d4ce None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:50.442246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e1864-37e7-42c5-a606-ebc9e517d4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:50.442517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e1864-37e7-42c5-a606-ebc9e517d4ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:50.443128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e1864-37e7-42c5-a606-ebc9e517d4ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:50.445016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e1864-37e7-42c5-a606-ebc9e517d4ce None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:50.447275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad27423-2775-4f77-a4af-da8e29fda6bf None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-f87e1864-37e7-42c5-a606-ebc9e517d4ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:50.454096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:50.455090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:50.456135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 491/984] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:50.460951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e1864-37e7-42c5-a606-ebc9e517d4ce None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:50.506161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:50.552708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad27423-2775-4f77-a4af-da8e29fda6bf None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:50.556312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:50.570202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e1864-37e7-42c5-a606-ebc9e517d4ce None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:50.589886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c50160-dce5-4d25-adda-a8f7f83c07d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:50.590518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c50160-dce5-4d25-adda-a8f7f83c07d5 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:50.590789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c50160-dce5-4d25-adda-a8f7f83c07d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:50.591082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c50160-dce5-4d25-adda-a8f7f83c07d5 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:50.591718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c50160-dce5-4d25-adda-a8f7f83c07d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:50.593067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c50160-dce5-4d25-adda-a8f7f83c07d5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:50.607329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de071c4c-ba18-4de3-83df-ede900263c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:50.607933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de071c4c-ba18-4de3-83df-ede900263c09 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:50.608214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de071c4c-ba18-4de3-83df-ede900263c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:50.608487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de071c4c-ba18-4de3-83df-ede900263c09 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:50.609136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de071c4c-ba18-4de3-83df-ede900263c09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:50.609443 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de071c4c-ba18-4de3-83df-ede900263c09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:50.610824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de071c4c-ba18-4de3-83df-ede900263c09 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:50.615160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 494/985] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:51:50 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:51:50.617107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 492/986] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:50.623239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c50160-dce5-4d25-adda-a8f7f83c07d5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:50.742369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c50160-dce5-4d25-adda-a8f7f83c07d5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:50.779078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b834ed-a255-4d77-b440-594c61c0f30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:50.779685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b834ed-a255-4d77-b440-594c61c0f30e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:50.779959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b834ed-a255-4d77-b440-594c61c0f30e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:50.780227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b834ed-a255-4d77-b440-594c61c0f30e None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:50.780848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b834ed-a255-4d77-b440-594c61c0f30e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:50.781197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b834ed-a255-4d77-b440-594c61c0f30e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:50.782573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b834ed-a255-4d77-b440-594c61c0f30e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:50.787721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 495/987] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:50.793901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de071c4c-ba18-4de3-83df-ede900263c09 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:50.892755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de071c4c-ba18-4de3-83df-ede900263c09 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:50.900037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:50.927517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60be3b-626c-40e5-b74a-307f6f806b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:50.928122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60be3b-626c-40e5-b74a-307f6f806b1f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:50.928399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60be3b-626c-40e5-b74a-307f6f806b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:50.929228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60be3b-626c-40e5-b74a-307f6f806b1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:50.929931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60be3b-626c-40e5-b74a-307f6f806b1f None admin] RBAC: Adding query filter params (interface=admin, service_id=9f6e39e8c60c4420a563498a94536855, region_id=RegionOne) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:50.930237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60be3b-626c-40e5-b74a-307f6f806b1f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:50.931616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60be3b-626c-40e5-b74a-307f6f806b1f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:50.936706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 493/988] 10.222.0.20 () {56 vars in 1249 bytes} [Wed Aug 10 09:51:50 2022] GET /identity/v3/endpoints?service_id=9f6e39e8c60c4420a563498a94536855&interface=admin®ion_id=RegionOne => generated 193 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:50.941957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b834ed-a255-4d77-b440-594c61c0f30e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:50.967021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:51.046516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b834ed-a255-4d77-b440-594c61c0f30e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:51.080008 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83ce73-471e-42f5-b548-218cd9bea377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:51.080614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83ce73-471e-42f5-b548-218cd9bea377 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:51.080889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83ce73-471e-42f5-b548-218cd9bea377 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:51.081159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83ce73-471e-42f5-b548-218cd9bea377 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:51.081757 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83ce73-471e-42f5-b548-218cd9bea377 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:51.083138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83ce73-471e-42f5-b548-218cd9bea377 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:51.095059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 496/989] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 10 09:51:50 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:51:51.578794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1058312-6377-438a-8b6f-74eb57b9a9d3 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:51.579361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1058312-6377-438a-8b6f-74eb57b9a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:51.579361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1058312-6377-438a-8b6f-74eb57b9a9d3 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:51.580726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 494/990] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:51.586360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d634fb-4972-4b07-91f1-4a86066c404f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:51.586778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d634fb-4972-4b07-91f1-4a86066c404f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:51.586999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d634fb-4972-4b07-91f1-4a86066c404f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:51.604413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e1d634fb-4972-4b07-91f1-4a86066c404f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:51.605570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e1d634fb-4972-4b07-91f1-4a86066c404f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:51.672733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 497/991] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:51.693489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1058312-6377-438a-8b6f-74eb57b9a9d3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:51.796851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1058312-6377-438a-8b6f-74eb57b9a9d3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:51.830684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0254-b271-4aa5-a493-78cccd39b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:51.831324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0254-b271-4aa5-a493-78cccd39b569 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:51.831596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0254-b271-4aa5-a493-78cccd39b569 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:51.831859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0254-b271-4aa5-a493-78cccd39b569 None admin] PATH_INFO: `/v3/services/image` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:51.832424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f0254-b271-4aa5-a493-78cccd39b569 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:51.833761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f0254-b271-4aa5-a493-78cccd39b569 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-523f0254-b271-4aa5-a493-78cccd39b569 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:51.842158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:51.843146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 09:51:51.843979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:51.847285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 495/992] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Aug 10 09:51:51 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:51.849059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d634fb-4972-4b07-91f1-4a86066c404f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:51.867435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:51.946045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d634fb-4972-4b07-91f1-4a86066c404f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:51.978792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaed9d-293a-4502-a49e-acfbd1adc0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:51.979618 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaed9d-293a-4502-a49e-acfbd1adc0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:51.980068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaed9d-293a-4502-a49e-acfbd1adc0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:51.980505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaed9d-293a-4502-a49e-acfbd1adc0c5 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:51.981291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaed9d-293a-4502-a49e-acfbd1adc0c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:51.981786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaed9d-293a-4502-a49e-acfbd1adc0c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:51.983323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaed9d-293a-4502-a49e-acfbd1adc0c5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:51.992842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 498/993] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:51.994444 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0254-b271-4aa5-a493-78cccd39b569 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:52.028479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:52.096834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0254-b271-4aa5-a493-78cccd39b569 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:52.131151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bfed7-4019-45cc-aaac-794458e23318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:52.132342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bfed7-4019-45cc-aaac-794458e23318 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:52.132650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bfed7-4019-45cc-aaac-794458e23318 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:52.132944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bfed7-4019-45cc-aaac-794458e23318 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:52.133578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bfed7-4019-45cc-aaac-794458e23318 None admin] RBAC: Adding query filter params (type=image) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:52.134014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bfed7-4019-45cc-aaac-794458e23318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:52.135267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bfed7-4019-45cc-aaac-794458e23318 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:52.140003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 496/994] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:52.661242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15545386-c847-451e-b7ca-0ff7bd81459c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:52.661751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15545386-c847-451e-b7ca-0ff7bd81459c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:52.661751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15545386-c847-451e-b7ca-0ff7bd81459c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:52.663806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 499/995] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:52.668662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac4030-67a9-4d07-b75d-cbe33e74fac8 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:52.669042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac4030-67a9-4d07-b75d-cbe33e74fac8 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:52.669235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac4030-67a9-4d07-b75d-cbe33e74fac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:52.687092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-1aac4030-67a9-4d07-b75d-cbe33e74fac8 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:52.688275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1aac4030-67a9-4d07-b75d-cbe33e74fac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:52.755131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 497/996] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:52.776495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15545386-c847-451e-b7ca-0ff7bd81459c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:52.858600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:52.878875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15545386-c847-451e-b7ca-0ff7bd81459c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:52.912614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ff63f-9d1b-403a-88a1-0a48743caa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:52.913715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ff63f-9d1b-403a-88a1-0a48743caa35 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:52.913986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ff63f-9d1b-403a-88a1-0a48743caa35 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:52.914257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ff63f-9d1b-403a-88a1-0a48743caa35 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:52.914835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ff63f-9d1b-403a-88a1-0a48743caa35 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:52.916318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ff63f-9d1b-403a-88a1-0a48743caa35 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e12ff63f-9d1b-403a-88a1-0a48743caa35 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:52.924814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:52.925864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 09:51:52.926706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:52.933868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 500/997] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:52.933997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac4030-67a9-4d07-b75d-cbe33e74fac8 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:52.960403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:53.033473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac4030-67a9-4d07-b75d-cbe33e74fac8 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:53.068320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2fcf1-9b64-4643-8905-fb953a2955e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:53.068926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2fcf1-9b64-4643-8905-fb953a2955e4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.069765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2fcf1-9b64-4643-8905-fb953a2955e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.070036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2fcf1-9b64-4643-8905-fb953a2955e4 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.070663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2fcf1-9b64-4643-8905-fb953a2955e4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:53.070973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2fcf1-9b64-4643-8905-fb953a2955e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:53.072357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2fcf1-9b64-4643-8905-fb953a2955e4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:53.073127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee539cf-d0ab-4088-a26e-c52b523dc701 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.073419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee539cf-d0ab-4088-a26e-c52b523dc701 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.073688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee539cf-d0ab-4088-a26e-c52b523dc701 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.075036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 501/998] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:53.078016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 498/999] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:53.079775 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272f996-06f2-4e16-bac5-997554dd0a1d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.080053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272f996-06f2-4e16-bac5-997554dd0a1d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.080321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272f996-06f2-4e16-bac5-997554dd0a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.083341 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2fcf1-9b64-4643-8905-fb953a2955e4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:53.098139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8272f996-06f2-4e16-bac5-997554dd0a1d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:53.099444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8272f996-06f2-4e16-bac5-997554dd0a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:53.183328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 502/1000] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:53.185139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:53.197279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2fcf1-9b64-4643-8905-fb953a2955e4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:53.208675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272f996-06f2-4e16-bac5-997554dd0a1d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:53.235204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd3064-1d71-4719-8ae5-16f98fdc7471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:53.236056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd3064-1d71-4719-8ae5-16f98fdc7471 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.236334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd3064-1d71-4719-8ae5-16f98fdc7471 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.236611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd3064-1d71-4719-8ae5-16f98fdc7471 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.237239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd3064-1d71-4719-8ae5-16f98fdc7471 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:53.237544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd3064-1d71-4719-8ae5-16f98fdc7471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:53.238933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd3064-1d71-4719-8ae5-16f98fdc7471 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:53.249329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 499/1001] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:53.250572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd3064-1d71-4719-8ae5-16f98fdc7471 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:53.295112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:53.320440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272f996-06f2-4e16-bac5-997554dd0a1d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:53.355358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd3064-1d71-4719-8ae5-16f98fdc7471 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:53.356825 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17842655-c988-4a76-a903-fbeca3b97a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:53.357449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17842655-c988-4a76-a903-fbeca3b97a91 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.357731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17842655-c988-4a76-a903-fbeca3b97a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.358002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17842655-c988-4a76-a903-fbeca3b97a91 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.358581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17842655-c988-4a76-a903-fbeca3b97a91 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:53.359965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17842655-c988-4a76-a903-fbeca3b97a91 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-17842655-c988-4a76-a903-fbeca3b97a91 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:53.369379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:53.370339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:53.371197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 503/1002] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 10 09:51:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:53.376360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17842655-c988-4a76-a903-fbeca3b97a91 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:53.394578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2ba06-bd1b-4d29-80a2-3465f17f2519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:53.395212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ba06-bd1b-4d29-80a2-3465f17f2519 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.398365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ba06-bd1b-4d29-80a2-3465f17f2519 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.398365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ba06-bd1b-4d29-80a2-3465f17f2519 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.398365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ba06-bd1b-4d29-80a2-3465f17f2519 None admin] RBAC: Adding query filter params (interface=public, service_id=3f1ea77cf6874b12961928fc23fb13f8, region_id=RegionOne) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:53.398365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ba06-bd1b-4d29-80a2-3465f17f2519 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:53.398365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2ba06-bd1b-4d29-80a2-3465f17f2519 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:53.403236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 500/1003] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 10 09:51:53 2022] GET /identity/v3/endpoints?service_id=3f1ea77cf6874b12961928fc23fb13f8&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:53.409767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2ba06-bd1b-4d29-80a2-3465f17f2519 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:53.432369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:53.497427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17842655-c988-4a76-a903-fbeca3b97a91 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:53.516508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2ba06-bd1b-4d29-80a2-3465f17f2519 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:53.534065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e493dc3-6d93-4f44-9297-3d22abc7cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:53.534669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e493dc3-6d93-4f44-9297-3d22abc7cce1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.534968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e493dc3-6d93-4f44-9297-3d22abc7cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.535247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e493dc3-6d93-4f44-9297-3d22abc7cce1 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.535866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e493dc3-6d93-4f44-9297-3d22abc7cce1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:53.536562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e493dc3-6d93-4f44-9297-3d22abc7cce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:53.537961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e493dc3-6d93-4f44-9297-3d22abc7cce1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:53.542838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 504/1004] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:53.548542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e493dc3-6d93-4f44-9297-3d22abc7cce1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:53.550512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70c615-c5e6-4612-b5fb-6a2096ada497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:53.551153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70c615-c5e6-4612-b5fb-6a2096ada497 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.551427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70c615-c5e6-4612-b5fb-6a2096ada497 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.551696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70c615-c5e6-4612-b5fb-6a2096ada497 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.552300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70c615-c5e6-4612-b5fb-6a2096ada497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:53.552589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:53.554228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70c615-c5e6-4612-b5fb-6a2096ada497 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:53.570982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 501/1005] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 10 09:51:53 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:51:53.660641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e493dc3-6d93-4f44-9297-3d22abc7cce1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:53.696883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bc0fe-43ed-4131-ae54-0c599126bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:53.697559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bc0fe-43ed-4131-ae54-0c599126bd2c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.697762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bc0fe-43ed-4131-ae54-0c599126bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.698033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bc0fe-43ed-4131-ae54-0c599126bd2c None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.698669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bc0fe-43ed-4131-ae54-0c599126bd2c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:53.698981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bc0fe-43ed-4131-ae54-0c599126bd2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:53.700365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bc0fe-43ed-4131-ae54-0c599126bd2c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:53.705693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 505/1006] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 10 09:51:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:53.711778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70c615-c5e6-4612-b5fb-6a2096ada497 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:53.802264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:53.820123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70c615-c5e6-4612-b5fb-6a2096ada497 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:53.854447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884146d-83b2-4543-9afa-5b9ca5b35649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:53.855081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884146d-83b2-4543-9afa-5b9ca5b35649 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:53.855363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884146d-83b2-4543-9afa-5b9ca5b35649 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:53.855638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884146d-83b2-4543-9afa-5b9ca5b35649 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:53.856290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884146d-83b2-4543-9afa-5b9ca5b35649 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:53.857655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884146d-83b2-4543-9afa-5b9ca5b35649 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:53.882978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 502/1007] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:51:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:51:54.158613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05fa1b-c75c-448f-8949-b35e94a94fd3 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:54.158789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05fa1b-c75c-448f-8949-b35e94a94fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:54.159096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05fa1b-c75c-448f-8949-b35e94a94fd3 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:54.160586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 506/1008] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:54.166079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221bdbb-4725-422e-a5bc-8968c6ccd339 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:54.166361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221bdbb-4725-422e-a5bc-8968c6ccd339 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:54.166632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221bdbb-4725-422e-a5bc-8968c6ccd339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:54.184287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a221bdbb-4725-422e-a5bc-8968c6ccd339 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:54.185448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a221bdbb-4725-422e-a5bc-8968c6ccd339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:54.256118 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 503/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:54.279020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05fa1b-c75c-448f-8949-b35e94a94fd3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:54.347000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:54.383733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05fa1b-c75c-448f-8949-b35e94a94fd3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:54.423368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd826855-d3f3-4090-8578-83d562c97477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:54.423969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd826855-d3f3-4090-8578-83d562c97477 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:54.424240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd826855-d3f3-4090-8578-83d562c97477 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:54.424505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd826855-d3f3-4090-8578-83d562c97477 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:54.425168 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd826855-d3f3-4090-8578-83d562c97477 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:54.426502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd826855-d3f3-4090-8578-83d562c97477 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:54.450992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 507/1010] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:51:54 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:55.594714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a7d4-4f3b-4b07-81b4-db3a12eb1ea4 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:55.595256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a7d4-4f3b-4b07-81b4-db3a12eb1ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:55.595431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a7d4-4f3b-4b07-81b4-db3a12eb1ea4 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:55.596714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 504/1011] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:55.602496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a00b2-733d-4fe2-a2e6-b7e752327256 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:55.602909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a00b2-733d-4fe2-a2e6-b7e752327256 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:55.603113 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a00b2-733d-4fe2-a2e6-b7e752327256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:55.620819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ae5a00b2-733d-4fe2-a2e6-b7e752327256 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:55.621989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ae5a00b2-733d-4fe2-a2e6-b7e752327256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:55.695075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 508/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:55 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:55.717416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a7d4-4f3b-4b07-81b4-db3a12eb1ea4 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:55.786815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:55.820044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a7d4-4f3b-4b07-81b4-db3a12eb1ea4 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:55.853554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24f37f-2084-4f08-b12b-e35e641b67c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:55.854174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24f37f-2084-4f08-b12b-e35e641b67c6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:55.854449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24f37f-2084-4f08-b12b-e35e641b67c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:55.854715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24f37f-2084-4f08-b12b-e35e641b67c6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:55.855312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24f37f-2084-4f08-b12b-e35e641b67c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:55.856657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24f37f-2084-4f08-b12b-e35e641b67c6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-9b24f37f-2084-4f08-b12b-e35e641b67c6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:55.865076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:55.866067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:55.866921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 505/1013] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:51:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:55.874024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a00b2-733d-4fe2-a2e6-b7e752327256 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:55.894320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67d1d2-5b99-4a47-a679-9cf67ee9df83 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:55.894611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67d1d2-5b99-4a47-a679-9cf67ee9df83 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:55.894902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67d1d2-5b99-4a47-a679-9cf67ee9df83 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:55.898981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 506/1014] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:55.901573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a44d94-eff8-46ca-98fe-cc999fd2319b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:55.901846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a44d94-eff8-46ca-98fe-cc999fd2319b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:55.902115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a44d94-eff8-46ca-98fe-cc999fd2319b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:55.920525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-b1a44d94-eff8-46ca-98fe-cc999fd2319b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:55.921683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b1a44d94-eff8-46ca-98fe-cc999fd2319b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:55.935696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:55.985299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a00b2-733d-4fe2-a2e6-b7e752327256 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:55.995400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 507/1015] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:55 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:56.014928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a44d94-eff8-46ca-98fe-cc999fd2319b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:56.026097 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730bc9a-073e-4e05-8e66-259214cbd4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:56.026705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730bc9a-073e-4e05-8e66-259214cbd4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:56.027401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730bc9a-073e-4e05-8e66-259214cbd4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:56.027675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730bc9a-073e-4e05-8e66-259214cbd4a2 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:56.028588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730bc9a-073e-4e05-8e66-259214cbd4a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:56.028890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730bc9a-073e-4e05-8e66-259214cbd4a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:56.031036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730bc9a-073e-4e05-8e66-259214cbd4a2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:56.042083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 509/1016] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:56.043508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730bc9a-073e-4e05-8e66-259214cbd4a2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:56.080086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:56.120602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a44d94-eff8-46ca-98fe-cc999fd2319b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:56.149643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730bc9a-073e-4e05-8e66-259214cbd4a2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:56.156729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac5da4-3a3c-4e74-a95b-b689a7417b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:56.157347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5da4-3a3c-4e74-a95b-b689a7417b01 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:56.157625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5da4-3a3c-4e74-a95b-b689a7417b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:56.157889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5da4-3a3c-4e74-a95b-b689a7417b01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:56.158988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac5da4-3a3c-4e74-a95b-b689a7417b01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:56.160405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac5da4-3a3c-4e74-a95b-b689a7417b01 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:56.161943 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 508/1017] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:56.167653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac5da4-3a3c-4e74-a95b-b689a7417b01 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:56.173011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:56.191260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21570ed9-96ba-496d-8886-177aaec590a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:56.191866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21570ed9-96ba-496d-8886-177aaec590a2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:56.192140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21570ed9-96ba-496d-8886-177aaec590a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:56.192404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21570ed9-96ba-496d-8886-177aaec590a2 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:56.193023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21570ed9-96ba-496d-8886-177aaec590a2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:56.193321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21570ed9-96ba-496d-8886-177aaec590a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:56.195197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21570ed9-96ba-496d-8886-177aaec590a2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:56.200248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 510/1018] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 10 09:51:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:56.205521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21570ed9-96ba-496d-8886-177aaec590a2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:56.272971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac5da4-3a3c-4e74-a95b-b689a7417b01 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:56.301236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:56.307487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363cd0b-b458-4fb5-aae4-50c1a0d9fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:56.308091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363cd0b-b458-4fb5-aae4-50c1a0d9fd22 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:56.308366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363cd0b-b458-4fb5-aae4-50c1a0d9fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:56.308854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363cd0b-b458-4fb5-aae4-50c1a0d9fd22 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:56.309260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363cd0b-b458-4fb5-aae4-50c1a0d9fd22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:56.310670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363cd0b-b458-4fb5-aae4-50c1a0d9fd22 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:56.311714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21570ed9-96ba-496d-8886-177aaec590a2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:56.338834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 509/1019] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:51:56 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:51:56.354269 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b180cfc-8d4a-4817-9743-916a680fe455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:56.354900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b180cfc-8d4a-4817-9743-916a680fe455 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:56.355253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b180cfc-8d4a-4817-9743-916a680fe455 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:56.355521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b180cfc-8d4a-4817-9743-916a680fe455 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:56.356201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b180cfc-8d4a-4817-9743-916a680fe455 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:56.357591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b180cfc-8d4a-4817-9743-916a680fe455 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:56.387000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 511/1020] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:51:56 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:51:56.463292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b938e11-3db5-4fda-9c7d-68453e100659 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:56.463620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b938e11-3db5-4fda-9c7d-68453e100659 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:56.463850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b938e11-3db5-4fda-9c7d-68453e100659 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:56.465455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 510/1021] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:56.471358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c651e-474a-4d30-bd89-3c292f67f372 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:56.471708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c651e-474a-4d30-bd89-3c292f67f372 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:56.471925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c651e-474a-4d30-bd89-3c292f67f372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:56.492031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0b9c651e-474a-4d30-bd89-3c292f67f372 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:56.493281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0b9c651e-474a-4d30-bd89-3c292f67f372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:56.528770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:56.576190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 512/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:56.598183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b938e11-3db5-4fda-9c7d-68453e100659 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:56.707725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b938e11-3db5-4fda-9c7d-68453e100659 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:56.741474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710de43-8883-4e64-bfde-fdef9395d0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:56.742111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710de43-8883-4e64-bfde-fdef9395d0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:56.742387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710de43-8883-4e64-bfde-fdef9395d0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:56.742656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710de43-8883-4e64-bfde-fdef9395d0e8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:56.743261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710de43-8883-4e64-bfde-fdef9395d0e8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:56.744889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710de43-8883-4e64-bfde-fdef9395d0e8 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-7710de43-8883-4e64-bfde-fdef9395d0e8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:56.753439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:56.754456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:56.755337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 511/1023] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Aug 10 09:51:56 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:56.760549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c651e-474a-4d30-bd89-3c292f67f372 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:56.859601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c651e-474a-4d30-bd89-3c292f67f372 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:56.894239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf654795-9c95-498f-aebd-2cb8c07a0f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:56.894861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf654795-9c95-498f-aebd-2cb8c07a0f6e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:56.895152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf654795-9c95-498f-aebd-2cb8c07a0f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:56.895428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf654795-9c95-498f-aebd-2cb8c07a0f6e None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:56.896066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf654795-9c95-498f-aebd-2cb8c07a0f6e None admin] RBAC: Adding query filter params (name=image) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:56.896368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf654795-9c95-498f-aebd-2cb8c07a0f6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:56.897753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf654795-9c95-498f-aebd-2cb8c07a0f6e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:56.902763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 513/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:56.908302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710de43-8883-4e64-bfde-fdef9395d0e8 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:56.960209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:57.015244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710de43-8883-4e64-bfde-fdef9395d0e8 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:57.049441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58bca6-190c-4b7d-9e1e-6e8bb50b2685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:57.050437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58bca6-190c-4b7d-9e1e-6e8bb50b2685 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:57.050939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58bca6-190c-4b7d-9e1e-6e8bb50b2685 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:57.051453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58bca6-190c-4b7d-9e1e-6e8bb50b2685 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:57.052373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58bca6-190c-4b7d-9e1e-6e8bb50b2685 None admin] RBAC: Adding query filter params (type=image) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:57.052876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58bca6-190c-4b7d-9e1e-6e8bb50b2685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:57.055024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58bca6-190c-4b7d-9e1e-6e8bb50b2685 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:57.060044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 512/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:57.065977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf654795-9c95-498f-aebd-2cb8c07a0f6e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:57.096854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:57.171112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf654795-9c95-498f-aebd-2cb8c07a0f6e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:57.204731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f6f48-e75c-4a1f-8e6f-ea9007bbbb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:57.205340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6f48-e75c-4a1f-8e6f-ea9007bbbb87 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:57.205622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6f48-e75c-4a1f-8e6f-ea9007bbbb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:57.205895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6f48-e75c-4a1f-8e6f-ea9007bbbb87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:57.206588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f6f48-e75c-4a1f-8e6f-ea9007bbbb87 None admin] RBAC: Adding query filter params (interface=public, service_id=c2f6ef0ce10f45d9aa739f7f533c8113, region_id=RegionOne) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:57.206895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f6f48-e75c-4a1f-8e6f-ea9007bbbb87 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:57.208281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f6f48-e75c-4a1f-8e6f-ea9007bbbb87 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:57.218510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 514/1026] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 10 09:51:57 2022] GET /identity/v3/endpoints?service_id=c2f6ef0ce10f45d9aa739f7f533c8113&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:57.219502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58bca6-190c-4b7d-9e1e-6e8bb50b2685 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:57.318427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58bca6-190c-4b7d-9e1e-6e8bb50b2685 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:57.331254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:57.353612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b927614-1505-4bb0-8ea2-f69a021c8c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:57.354211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b927614-1505-4bb0-8ea2-f69a021c8c28 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:57.354488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b927614-1505-4bb0-8ea2-f69a021c8c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:57.354760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b927614-1505-4bb0-8ea2-f69a021c8c28 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:57.355386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b927614-1505-4bb0-8ea2-f69a021c8c28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:57.356747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b927614-1505-4bb0-8ea2-f69a021c8c28 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:57.363286 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 513/1027] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 10 09:51:57 2022] GET /identity/v3/services => generated 1957 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:51:58.419359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e6e8f-7d0f-4e40-bec0-0b914861119b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:58.419797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e6e8f-7d0f-4e40-bec0-0b914861119b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:58.419797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e6e8f-7d0f-4e40-bec0-0b914861119b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:58.421225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 515/1028] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:58.426193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3927d36-a6db-4efe-8b2b-00cea425db44 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:58.426588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3927d36-a6db-4efe-8b2b-00cea425db44 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:58.426760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3927d36-a6db-4efe-8b2b-00cea425db44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:58.444810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f3927d36-a6db-4efe-8b2b-00cea425db44 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:58.445971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f3927d36-a6db-4efe-8b2b-00cea425db44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:58.515273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 514/1029] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:51:58.537002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c87214-93b5-4f74-a3d6-00f3dec262f1 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:58.537280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c87214-93b5-4f74-a3d6-00f3dec262f1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:58.537554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c87214-93b5-4f74-a3d6-00f3dec262f1 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:58.539698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 516/1030] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:51:58.539957 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3927d36-a6db-4efe-8b2b-00cea425db44 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:58.544357 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a09703-1a02-48e0-8553-f1b1b963dc0d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:58.544641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a09703-1a02-48e0-8553-f1b1b963dc0d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:58.544912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a09703-1a02-48e0-8553-f1b1b963dc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:58.562818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-77a09703-1a02-48e0-8553-f1b1b963dc0d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:58.564016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-77a09703-1a02-48e0-8553-f1b1b963dc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:58.632563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 517/1031] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:58.640479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3927d36-a6db-4efe-8b2b-00cea425db44 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:58.648901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:58.655564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a09703-1a02-48e0-8553-f1b1b963dc0d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:58.678037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d95832-0778-4821-be2e-84f17d41f0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:58.678610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d95832-0778-4821-be2e-84f17d41f0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:58.678880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d95832-0778-4821-be2e-84f17d41f0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:58.679186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d95832-0778-4821-be2e-84f17d41f0b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:58.679786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:58.680279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d95832-0778-4821-be2e-84f17d41f0b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:58.681707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d95832-0778-4821-be2e-84f17d41f0b2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:58.687093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 515/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:58.688837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d95832-0778-4821-be2e-84f17d41f0b2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:58.757017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a09703-1a02-48e0-8553-f1b1b963dc0d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:58.792538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f96c6-131e-4cea-aeb7-8fbf28bdfabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:58.793262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f96c6-131e-4cea-aeb7-8fbf28bdfabd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:58.793548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f96c6-131e-4cea-aeb7-8fbf28bdfabd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:58.793835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f96c6-131e-4cea-aeb7-8fbf28bdfabd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:58.793935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d95832-0778-4821-be2e-84f17d41f0b2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:58.804625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f96c6-131e-4cea-aeb7-8fbf28bdfabd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ea4f96c6-131e-4cea-aeb7-8fbf28bdfabd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:51:58.809249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:58.810358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:51:58.811748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 518/1033] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 10 09:51:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:58.816064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f96c6-131e-4cea-aeb7-8fbf28bdfabd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:58.833826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af7aa9-99ae-4ff3-bd86-f598bd1a44af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:58.834429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af7aa9-99ae-4ff3-bd86-f598bd1a44af None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:58.835270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af7aa9-99ae-4ff3-bd86-f598bd1a44af None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:58.835541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af7aa9-99ae-4ff3-bd86-f598bd1a44af None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:58.836171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af7aa9-99ae-4ff3-bd86-f598bd1a44af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:58.837582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af7aa9-99ae-4ff3-bd86-f598bd1a44af None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:58.860459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 516/1034] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:51:58 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:51:58.903376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:58.924077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f96c6-131e-4cea-aeb7-8fbf28bdfabd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:58.961284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a940e-a551-4440-80ea-f16d36b8faaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:58.961809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a940e-a551-4440-80ea-f16d36b8faaf None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:58.962082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a940e-a551-4440-80ea-f16d36b8faaf None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:58.962350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a940e-a551-4440-80ea-f16d36b8faaf None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:58.963125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a940e-a551-4440-80ea-f16d36b8faaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:58.963428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a940e-a551-4440-80ea-f16d36b8faaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:58.964863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a940e-a551-4440-80ea-f16d36b8faaf None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:58.973793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 519/1035] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 10 09:51:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:58.979754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af7aa9-99ae-4ff3-bd86-f598bd1a44af None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:59.012390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:59.085212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af7aa9-99ae-4ff3-bd86-f598bd1a44af None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:59.122839 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402bc91-c296-40bd-956b-225cbbcb791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:59.124297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402bc91-c296-40bd-956b-225cbbcb791a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.124995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402bc91-c296-40bd-956b-225cbbcb791a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.125716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402bc91-c296-40bd-956b-225cbbcb791a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.142013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402bc91-c296-40bd-956b-225cbbcb791a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:59.143744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402bc91-c296-40bd-956b-225cbbcb791a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-e402bc91-c296-40bd-956b-225cbbcb791a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:51:59.158363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:59.159566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:51:59.160651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:51:59.160651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:51:59.160651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:59.162079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 517/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:59.168373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a940e-a551-4440-80ea-f16d36b8faaf None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:59.269882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a940e-a551-4440-80ea-f16d36b8faaf None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:59.302711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f302d9-f4b9-4fb9-860c-788d7bb38102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:59.303894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f302d9-f4b9-4fb9-860c-788d7bb38102 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.304179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f302d9-f4b9-4fb9-860c-788d7bb38102 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.304440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f302d9-f4b9-4fb9-860c-788d7bb38102 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.305065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f302d9-f4b9-4fb9-860c-788d7bb38102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:59.305360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f302d9-f4b9-4fb9-860c-788d7bb38102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:59.306695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f302d9-f4b9-4fb9-860c-788d7bb38102 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:59.318426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 520/1037] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:59.323876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402bc91-c296-40bd-956b-225cbbcb791a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:59.380484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b55fc-8651-44aa-b3dd-21e2c34e3576 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.380796 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b55fc-8651-44aa-b3dd-21e2c34e3576 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.381019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b55fc-8651-44aa-b3dd-21e2c34e3576 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.382381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 521/1038] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:51:59.387232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f591b-7963-4c57-af34-54cff0079c58 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.387514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f591b-7963-4c57-af34-54cff0079c58 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.387787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f591b-7963-4c57-af34-54cff0079c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.394875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:59.411562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1d2f591b-7963-4c57-af34-54cff0079c58 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:51:59.412506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:59.413682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1d2f591b-7963-4c57-af34-54cff0079c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:51:59.434169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402bc91-c296-40bd-956b-225cbbcb791a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:59.470973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9539b-d63d-40e8-9bb8-736ebcdb6096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:59.471602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9539b-d63d-40e8-9bb8-736ebcdb6096 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.471881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9539b-d63d-40e8-9bb8-736ebcdb6096 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.472149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9539b-d63d-40e8-9bb8-736ebcdb6096 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.480291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 522/1039] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:51:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:51:59.480840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9539b-d63d-40e8-9bb8-736ebcdb6096 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:59.481974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9539b-d63d-40e8-9bb8-736ebcdb6096 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-e4d9539b-d63d-40e8-9bb8-736ebcdb6096 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:51:59.492887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:59.494114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:59.495568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 518/1040] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 10 09:51:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:51:59.499889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f591b-7963-4c57-af34-54cff0079c58 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:59.504730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9539b-d63d-40e8-9bb8-736ebcdb6096 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:59.606226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f591b-7963-4c57-af34-54cff0079c58 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:59.607740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9539b-d63d-40e8-9bb8-736ebcdb6096 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:59.640994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc952933-29d9-4fa9-8ae6-b9a4e27a25f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:59.641645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc952933-29d9-4fa9-8ae6-b9a4e27a25f8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.641921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc952933-29d9-4fa9-8ae6-b9a4e27a25f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.642197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc952933-29d9-4fa9-8ae6-b9a4e27a25f8 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.642903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc952933-29d9-4fa9-8ae6-b9a4e27a25f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:59.643229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc952933-29d9-4fa9-8ae6-b9a4e27a25f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:59.643431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1bb2e-98ae-4eff-8a3d-f4d933586cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:59.644054 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1bb2e-98ae-4eff-8a3d-f4d933586cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.644333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1bb2e-98ae-4eff-8a3d-f4d933586cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.644599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1bb2e-98ae-4eff-8a3d-f4d933586cc7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.644694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc952933-29d9-4fa9-8ae6-b9a4e27a25f8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:59.645176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1bb2e-98ae-4eff-8a3d-f4d933586cc7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:59.646473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:59.646579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1bb2e-98ae-4eff-8a3d-f4d933586cc7 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-5af1bb2e-98ae-4eff-8a3d-f4d933586cc7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:51:59.656114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:51:59.659382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:51:59.660404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 519/1041] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Aug 10 09:51:59 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:51:59.662877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1bb2e-98ae-4eff-8a3d-f4d933586cc7 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:59.669884 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 523/1042] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 10 09:51:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:59.672375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:59.675437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc952933-29d9-4fa9-8ae6-b9a4e27a25f8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:59.766933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1bb2e-98ae-4eff-8a3d-f4d933586cc7 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:59.782279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc952933-29d9-4fa9-8ae6-b9a4e27a25f8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:59.800339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dddb2-a326-465f-bbc7-50c3b448a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:59.800937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dddb2-a326-465f-bbc7-50c3b448a240 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.801234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dddb2-a326-465f-bbc7-50c3b448a240 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.801505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dddb2-a326-465f-bbc7-50c3b448a240 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.802126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0dddb2-a326-465f-bbc7-50c3b448a240 None admin] RBAC: Adding query filter params (name=image) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:59.802424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0dddb2-a326-465f-bbc7-50c3b448a240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:59.804430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0dddb2-a326-465f-bbc7-50c3b448a240 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:59.809132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 520/1043] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:51:59.814659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dddb2-a326-465f-bbc7-50c3b448a240 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:51:59.824122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acebf5-9973-4cc0-8d96-2854adda7b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:59.824729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acebf5-9973-4cc0-8d96-2854adda7b6a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.825014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acebf5-9973-4cc0-8d96-2854adda7b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.825286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acebf5-9973-4cc0-8d96-2854adda7b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.882707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:59.885316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:51:59.897257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acebf5-9973-4cc0-8d96-2854adda7b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:59.898704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acebf5-9973-4cc0-8d96-2854adda7b6a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:59.913953 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dddb2-a326-465f-bbc7-50c3b448a240 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:51:59.946777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acaaf7-7fb9-4d5d-b419-1e9fd0aab6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:51:59.947406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acaaf7-7fb9-4d5d-b419-1e9fd0aab6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:51:59.947688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acaaf7-7fb9-4d5d-b419-1e9fd0aab6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:51:59.947973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acaaf7-7fb9-4d5d-b419-1e9fd0aab6a0 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:51:59.948573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acaaf7-7fb9-4d5d-b419-1e9fd0aab6a0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:51:59.948872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acaaf7-7fb9-4d5d-b419-1e9fd0aab6a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:51:59.950226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acaaf7-7fb9-4d5d-b419-1e9fd0aab6a0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:51:59.959539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 521/1044] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:51:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:51:59.960559 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acaaf7-7fb9-4d5d-b419-1e9fd0aab6a0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:00.009359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 524/1045] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:51:59 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 09:52:00.015504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acebf5-9973-4cc0-8d96-2854adda7b6a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:00.066792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acaaf7-7fb9-4d5d-b419-1e9fd0aab6a0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:00.101070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e1fa3-25f9-4118-9799-f7693eb3581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:00.101676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e1fa3-25f9-4118-9799-f7693eb3581e None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:00.101951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e1fa3-25f9-4118-9799-f7693eb3581e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:00.102256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e1fa3-25f9-4118-9799-f7693eb3581e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:00.102864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e1fa3-25f9-4118-9799-f7693eb3581e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:00.104267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e1fa3-25f9-4118-9799-f7693eb3581e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:00.109780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:00.119348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:00.126883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 522/1046] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:51:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:52:00.130569 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acebf5-9973-4cc0-8d96-2854adda7b6a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:00.177384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c35b50-aa79-4318-ae3b-540f7209e096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:00.177944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c35b50-aa79-4318-ae3b-540f7209e096 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:00.178217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c35b50-aa79-4318-ae3b-540f7209e096 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:00.178526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c35b50-aa79-4318-ae3b-540f7209e096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:00.190132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c35b50-aa79-4318-ae3b-540f7209e096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:00.191751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c35b50-aa79-4318-ae3b-540f7209e096 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-21c35b50-aa79-4318-ae3b-540f7209e096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:52:00.207952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:00.208977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:52:00.210351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:52:00.210351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:00.210351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:00.214527 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 525/1047] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:52:00.215783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e1fa3-25f9-4118-9799-f7693eb3581e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:00.323795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e1fa3-25f9-4118-9799-f7693eb3581e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:00.357266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf3ff0-4295-48d7-8d95-d1e82e3ee821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:00.357865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf3ff0-4295-48d7-8d95-d1e82e3ee821 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:00.358137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf3ff0-4295-48d7-8d95-d1e82e3ee821 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:00.358402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf3ff0-4295-48d7-8d95-d1e82e3ee821 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:00.359064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf3ff0-4295-48d7-8d95-d1e82e3ee821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:00.359362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf3ff0-4295-48d7-8d95-d1e82e3ee821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:00.360705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf3ff0-4295-48d7-8d95-d1e82e3ee821 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:00.372851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 523/1048] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:00.378351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c35b50-aa79-4318-ae3b-540f7209e096 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:00.488009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c35b50-aa79-4318-ae3b-540f7209e096 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:00.522875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636c106-28fa-4e10-9bee-9a46b0eb653c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:00.523523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636c106-28fa-4e10-9bee-9a46b0eb653c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:00.523801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636c106-28fa-4e10-9bee-9a46b0eb653c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:00.524068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636c106-28fa-4e10-9bee-9a46b0eb653c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:00.530762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636c106-28fa-4e10-9bee-9a46b0eb653c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:00.532241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636c106-28fa-4e10-9bee-9a46b0eb653c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:00.535067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6636c106-28fa-4e10-9bee-9a46b0eb653c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:52:00.544880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:00.545995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:00.547126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:52:00.547126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:52:00.547126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:52:00.547126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:52:00.547126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:52:00.547126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:52:00.547126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:52:00.547126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:00.549302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 526/1049] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:52:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:00.555434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf3ff0-4295-48d7-8d95-d1e82e3ee821 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:00.606838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:00.660431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf3ff0-4295-48d7-8d95-d1e82e3ee821 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:00.694710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308ad74-bb74-492d-9294-2f45949ef221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:00.695336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308ad74-bb74-492d-9294-2f45949ef221 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:00.695612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308ad74-bb74-492d-9294-2f45949ef221 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:00.695878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308ad74-bb74-492d-9294-2f45949ef221 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:00.696557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308ad74-bb74-492d-9294-2f45949ef221 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:00.696859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308ad74-bb74-492d-9294-2f45949ef221 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:00.698269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308ad74-bb74-492d-9294-2f45949ef221 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:00.714921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 524/1050] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:52:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:00.716142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636c106-28fa-4e10-9bee-9a46b0eb653c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:00.815241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636c106-28fa-4e10-9bee-9a46b0eb653c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:00.848057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284af0b-5d23-4572-80d8-48b0c591b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:00.848654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284af0b-5d23-4572-80d8-48b0c591b596 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:00.848927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284af0b-5d23-4572-80d8-48b0c591b596 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:00.849190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284af0b-5d23-4572-80d8-48b0c591b596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:00.849900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284af0b-5d23-4572-80d8-48b0c591b596 None admin] RBAC: Adding query filter params (role.id=b0c99aa3196841039c3bb72b951c33fb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=904f7f036565419b9fdecc5e47acb45c) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:00.850235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284af0b-5d23-4572-80d8-48b0c591b596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:00.851663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284af0b-5d23-4572-80d8-48b0c591b596 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:00.862998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 527/1051] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:00 2022] GET /identity/v3/role_assignments?user.id=904f7f036565419b9fdecc5e47acb45c&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=b0c99aa3196841039c3bb72b951c33fb => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:00.982274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4be14-476b-4825-97be-15bbb2117dfa None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:00.982719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4be14-476b-4825-97be-15bbb2117dfa None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:00.983126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4be14-476b-4825-97be-15bbb2117dfa None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:00.984605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 525/1052] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:00.990181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0e70-43db-4b5b-b8c9-847b54f3e770 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:00.990471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0e70-43db-4b5b-b8c9-847b54f3e770 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:00.991341 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0e70-43db-4b5b-b8c9-847b54f3e770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:01.010229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c62d0e70-43db-4b5b-b8c9-847b54f3e770 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:01.011523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c62d0e70-43db-4b5b-b8c9-847b54f3e770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:01.071403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:01.083021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 528/1053] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:01.103913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4be14-476b-4825-97be-15bbb2117dfa None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:01.208522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4be14-476b-4825-97be-15bbb2117dfa None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:01.219583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:01.244908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82eae1-ab91-4f55-8524-6a9462ec24f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:01.245683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82eae1-ab91-4f55-8524-6a9462ec24f1 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:01.246050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82eae1-ab91-4f55-8524-6a9462ec24f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:01.246413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82eae1-ab91-4f55-8524-6a9462ec24f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:01.257202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82eae1-ab91-4f55-8524-6a9462ec24f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-2b82eae1-ab91-4f55-8524-6a9462ec24f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:01.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:01.263027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:01.264051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:01.265794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 526/1054] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:52:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:01.272112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d0e70-43db-4b5b-b8c9-847b54f3e770 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:01.376255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d0e70-43db-4b5b-b8c9-847b54f3e770 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:01.416229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8d136-9859-40c3-8862-5513e845bedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:01.416868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8d136-9859-40c3-8862-5513e845bedd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:01.417149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8d136-9859-40c3-8862-5513e845bedd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:01.417418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8d136-9859-40c3-8862-5513e845bedd None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:01.418085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8d136-9859-40c3-8862-5513e845bedd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:01.418391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8d136-9859-40c3-8862-5513e845bedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:01.419780 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8d136-9859-40c3-8862-5513e845bedd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:01.433224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 529/1055] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:52:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:01.434723 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82eae1-ab91-4f55-8524-6a9462ec24f1 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:01.532652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82eae1-ab91-4f55-8524-6a9462ec24f1 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:01.566089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecca163-15ee-4e46-8703-83dfdb17ba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:01.566814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecca163-15ee-4e46-8703-83dfdb17ba62 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:01.567120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecca163-15ee-4e46-8703-83dfdb17ba62 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:01.567386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecca163-15ee-4e46-8703-83dfdb17ba62 None admin] PATH_INFO: `/v3/users/45004a2c55584f3b9686cdbdb166fc83` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:01.580508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecca163-15ee-4e46-8703-83dfdb17ba62 None admin] RBAC: Authorizing `identity:get_user(user_id=45004a2c55584f3b9686cdbdb166fc83)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:01.581947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecca163-15ee-4e46-8703-83dfdb17ba62 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:01.583808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 527/1056] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:01 2022] GET /identity/v3/users/45004a2c55584f3b9686cdbdb166fc83 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:01.590070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8d136-9859-40c3-8862-5513e845bedd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:01.649383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:01.694923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8d136-9859-40c3-8862-5513e845bedd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:01.729353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1a2c0-b184-48c4-84b4-9187e14f1640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:01.729955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1a2c0-b184-48c4-84b4-9187e14f1640 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:01.730230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1a2c0-b184-48c4-84b4-9187e14f1640 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:01.730495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1a2c0-b184-48c4-84b4-9187e14f1640 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:01.798417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1a2c0-b184-48c4-84b4-9187e14f1640 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:01.799894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1a2c0-b184-48c4-84b4-9187e14f1640 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:01.853340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:01.903010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 530/1057] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:52:01.904813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecca163-15ee-4e46-8703-83dfdb17ba62 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:01.933046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:02.007826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecca163-15ee-4e46-8703-83dfdb17ba62 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:02.042072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9334c8c-dea9-4584-890e-472c8bbda37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:02.042711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9334c8c-dea9-4584-890e-472c8bbda37b None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:02.042988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9334c8c-dea9-4584-890e-472c8bbda37b None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:02.043263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9334c8c-dea9-4584-890e-472c8bbda37b None admin] PATH_INFO: `/v3/projects/35f4047759834ed49d1e59318821ad1e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:02.044356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9334c8c-dea9-4584-890e-472c8bbda37b None admin] RBAC: Authorizing `identity:get_project(project_id=35f4047759834ed49d1e59318821ad1e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:02.045770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9334c8c-dea9-4584-890e-472c8bbda37b None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:02.052330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 528/1058] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:01 2022] GET /identity/v3/projects/35f4047759834ed49d1e59318821ad1e => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:02.053854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1a2c0-b184-48c4-84b4-9187e14f1640 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:02.157914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1a2c0-b184-48c4-84b4-9187e14f1640 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:02.167699 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bdfc5-7fb7-40eb-af5e-da8cf4d202f6 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:02.167980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bdfc5-7fb7-40eb-af5e-da8cf4d202f6 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:02.168243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bdfc5-7fb7-40eb-af5e-da8cf4d202f6 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:02.169700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 529/1059] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:52:02.174349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf46385-3410-487e-85c2-74833b193f69 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:02.174823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf46385-3410-487e-85c2-74833b193f69 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:02.175134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf46385-3410-487e-85c2-74833b193f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:02.192101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86842a-86d2-4717-a452-ece4f6a07d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:02.192481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-9cf46385-3410-487e-85c2-74833b193f69 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:02.192707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86842a-86d2-4717-a452-ece4f6a07d33 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:02.192984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86842a-86d2-4717-a452-ece4f6a07d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:02.193251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86842a-86d2-4717-a452-ece4f6a07d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:02.193639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9cf46385-3410-487e-85c2-74833b193f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:02.194490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86842a-86d2-4717-a452-ece4f6a07d33 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=35f4047759834ed49d1e59318821ad1e, user.id=45004a2c55584f3b9686cdbdb166fc83) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:02.194798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86842a-86d2-4717-a452-ece4f6a07d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:02.196236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86842a-86d2-4717-a452-ece4f6a07d33 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:02.201635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 531/1060] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:02 2022] GET /identity/v3/role_assignments?user.id=45004a2c55584f3b9686cdbdb166fc83&scope.project.id=35f4047759834ed49d1e59318821ad1e&role.id=8417343e9ef64991bcf51bb9692da102 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:02.278014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 530/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:02.300085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86842a-86d2-4717-a452-ece4f6a07d33 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:02.326024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:02.415436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86842a-86d2-4717-a452-ece4f6a07d33 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:02.449762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3452676-e897-42e0-beb7-621ec9ca43f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:02.450365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3452676-e897-42e0-beb7-621ec9ca43f3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:02.450640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3452676-e897-42e0-beb7-621ec9ca43f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:02.450907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3452676-e897-42e0-beb7-621ec9ca43f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:02.518239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3452676-e897-42e0-beb7-621ec9ca43f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:02.519708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3452676-e897-42e0-beb7-621ec9ca43f3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:02.539705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:02.621712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 532/1062] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:52:02.623781 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf46385-3410-487e-85c2-74833b193f69 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:02.652288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:02.726582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf46385-3410-487e-85c2-74833b193f69 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:02.760979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590a4e6-fb5e-4af8-8306-00d8554d7a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:02.761602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a4e6-fb5e-4af8-8306-00d8554d7a80 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:02.761876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a4e6-fb5e-4af8-8306-00d8554d7a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:02.762145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a4e6-fb5e-4af8-8306-00d8554d7a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:02.772117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590a4e6-fb5e-4af8-8306-00d8554d7a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:02.773547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590a4e6-fb5e-4af8-8306-00d8554d7a80 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-3590a4e6-fb5e-4af8-8306-00d8554d7a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:52:02.788108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:02.789119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:52:02.790031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:52:02.790031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:02.790031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:02.790031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 531/1063] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:02.795203 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3452676-e897-42e0-beb7-621ec9ca43f3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:02.861968 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b72dc-7f45-409c-b9a6-fcbf7f1d9f19 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:02.862254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b72dc-7f45-409c-b9a6-fcbf7f1d9f19 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:02.862535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b72dc-7f45-409c-b9a6-fcbf7f1d9f19 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:02.863897 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 532/1064] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:52:02.869624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570ac52-e617-46f5-bd6d-7f2991b03fb3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:02.869906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570ac52-e617-46f5-bd6d-7f2991b03fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:02.870181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570ac52-e617-46f5-bd6d-7f2991b03fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:02.887979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a570ac52-e617-46f5-bd6d-7f2991b03fb3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:02.889159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a570ac52-e617-46f5-bd6d-7f2991b03fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:02.898768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3452676-e897-42e0-beb7-621ec9ca43f3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:02.922535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:02.937924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6125ed-31c5-4848-8113-0976b5cc384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:02.938717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6125ed-31c5-4848-8113-0976b5cc384c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:02.938845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6125ed-31c5-4848-8113-0976b5cc384c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:02.939118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6125ed-31c5-4848-8113-0976b5cc384c None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:02.939761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6125ed-31c5-4848-8113-0976b5cc384c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:02.940068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6125ed-31c5-4848-8113-0976b5cc384c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:02.941475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6125ed-31c5-4848-8113-0976b5cc384c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:02.960235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 533/1065] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:02.960435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 533/1066] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:52:02.983100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6125ed-31c5-4848-8113-0976b5cc384c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:03.087065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6125ed-31c5-4848-8113-0976b5cc384c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:03.125530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24362845-ecd4-4467-a04b-a48b2ae310dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:03.126159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24362845-ecd4-4467-a04b-a48b2ae310dd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:03.126437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24362845-ecd4-4467-a04b-a48b2ae310dd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:03.126704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24362845-ecd4-4467-a04b-a48b2ae310dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:03.139002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24362845-ecd4-4467-a04b-a48b2ae310dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-24362845-ecd4-4467-a04b-a48b2ae310dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:03.143768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:03.144876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:03.145886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 534/1067] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 10 09:52:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:03.151872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570ac52-e617-46f5-bd6d-7f2991b03fb3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:03.163840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:03.261648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570ac52-e617-46f5-bd6d-7f2991b03fb3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:03.296011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c023a-0b0f-4214-8b39-2c31620026b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:03.296636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c023a-0b0f-4214-8b39-2c31620026b8 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:03.296914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c023a-0b0f-4214-8b39-2c31620026b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:03.297186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c023a-0b0f-4214-8b39-2c31620026b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:03.297861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c023a-0b0f-4214-8b39-2c31620026b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:03.298165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c023a-0b0f-4214-8b39-2c31620026b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:03.299574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c023a-0b0f-4214-8b39-2c31620026b8 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:03.308046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 534/1068] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 10 09:52:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:03.314007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24362845-ecd4-4467-a04b-a48b2ae310dd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:03.354770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:03.415174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24362845-ecd4-4467-a04b-a48b2ae310dd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:03.449399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a081ccf-6362-4d4f-8f2c-10219765bc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:03.450018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a081ccf-6362-4d4f-8f2c-10219765bc24 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:03.450292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a081ccf-6362-4d4f-8f2c-10219765bc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:03.450561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a081ccf-6362-4d4f-8f2c-10219765bc24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:03.461172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a081ccf-6362-4d4f-8f2c-10219765bc24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:03.462582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a081ccf-6362-4d4f-8f2c-10219765bc24 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-3a081ccf-6362-4d4f-8f2c-10219765bc24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:52:03.475758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:03.476721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:52:03.478027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:52:03.478027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:03.478027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:03.478027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 535/1069] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:03.482478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c023a-0b0f-4214-8b39-2c31620026b8 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:03.549309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:03.579779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c023a-0b0f-4214-8b39-2c31620026b8 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:03.611869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad2318-00a2-43ab-b22e-c58e63055636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:03.612466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad2318-00a2-43ab-b22e-c58e63055636 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:03.612746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad2318-00a2-43ab-b22e-c58e63055636 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:03.613018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad2318-00a2-43ab-b22e-c58e63055636 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:03.613652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad2318-00a2-43ab-b22e-c58e63055636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:03.613951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad2318-00a2-43ab-b22e-c58e63055636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:03.615336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad2318-00a2-43ab-b22e-c58e63055636 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:03.627728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 535/1070] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:03.633769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a081ccf-6362-4d4f-8f2c-10219765bc24 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:03.721707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:03.731096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a081ccf-6362-4d4f-8f2c-10219765bc24 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:03.763902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d89fd-34ca-4947-b452-c73127f1f063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:03.764526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d89fd-34ca-4947-b452-c73127f1f063 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:03.764806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d89fd-34ca-4947-b452-c73127f1f063 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:03.765080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d89fd-34ca-4947-b452-c73127f1f063 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:03.772856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d89fd-34ca-4947-b452-c73127f1f063 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:03.774292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d89fd-34ca-4947-b452-c73127f1f063 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-669d89fd-34ca-4947-b452-c73127f1f063 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:52:03.784943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:03.786004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:03.787413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 536/1071] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 10 09:52:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:52:03.791736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad2318-00a2-43ab-b22e-c58e63055636 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:03.887783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad2318-00a2-43ab-b22e-c58e63055636 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:03.914900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:03.921338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba6698-1a3b-4e53-81d9-68f5ab35ec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:03.921934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba6698-1a3b-4e53-81d9-68f5ab35ec35 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:03.922209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba6698-1a3b-4e53-81d9-68f5ab35ec35 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:03.922474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba6698-1a3b-4e53-81d9-68f5ab35ec35 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:03.923227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba6698-1a3b-4e53-81d9-68f5ab35ec35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:03.923528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba6698-1a3b-4e53-81d9-68f5ab35ec35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:03.924935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba6698-1a3b-4e53-81d9-68f5ab35ec35 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:03.941848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 536/1072] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 10 09:52:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:03.948099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d89fd-34ca-4947-b452-c73127f1f063 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:04.043942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d89fd-34ca-4947-b452-c73127f1f063 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:04.076357 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacd53e-3277-4921-84c0-b639a2558c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:04.076970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd53e-3277-4921-84c0-b639a2558c57 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.077245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd53e-3277-4921-84c0-b639a2558c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.077510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd53e-3277-4921-84c0-b639a2558c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.087519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacd53e-3277-4921-84c0-b639a2558c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:04.088945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacd53e-3277-4921-84c0-b639a2558c57 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-2cacd53e-3277-4921-84c0-b639a2558c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:52:04.102188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:04.103306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:52:04.104396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:52:04.104396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:04.104396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:04.107100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 537/1073] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:04.108923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba6698-1a3b-4e53-81d9-68f5ab35ec35 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:04.192887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ad27-b58b-4090-a63c-7bb58e8e4129 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.193215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ad27-b58b-4090-a63c-7bb58e8e4129 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.193559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ad27-b58b-4090-a63c-7bb58e8e4129 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.195601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 538/1074] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:52:04.200504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cbfce-7a1e-4c75-9e6b-9187a534bbd6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.200786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cbfce-7a1e-4c75-9e6b-9187a534bbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.201057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cbfce-7a1e-4c75-9e6b-9187a534bbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.208425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba6698-1a3b-4e53-81d9-68f5ab35ec35 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:04.209329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:04.219169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0f7cbfce-7a1e-4c75-9e6b-9187a534bbd6 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:04.220350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0f7cbfce-7a1e-4c75-9e6b-9187a534bbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:04.242683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac25c98-a4d9-4d49-a875-931a9b1a605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:04.243333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac25c98-a4d9-4d49-a875-931a9b1a605d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.243612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac25c98-a4d9-4d49-a875-931a9b1a605d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.244150 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac25c98-a4d9-4d49-a875-931a9b1a605d None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.244783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac25c98-a4d9-4d49-a875-931a9b1a605d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:04.245083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac25c98-a4d9-4d49-a875-931a9b1a605d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:04.246433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac25c98-a4d9-4d49-a875-931a9b1a605d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:04.257248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 537/1075] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:04.262767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac25c98-a4d9-4d49-a875-931a9b1a605d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:04.287099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 539/1076] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:52:04.310079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cbfce-7a1e-4c75-9e6b-9187a534bbd6 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:04.311058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:04.363994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac25c98-a4d9-4d49-a875-931a9b1a605d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:04.396613 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce1e8-e66c-47ff-93b0-e986eb596d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:04.397231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce1e8-e66c-47ff-93b0-e986eb596d9c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.397504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce1e8-e66c-47ff-93b0-e986eb596d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.397770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce1e8-e66c-47ff-93b0-e986eb596d9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.404379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ce1e8-e66c-47ff-93b0-e986eb596d9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:04.405822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ce1e8-e66c-47ff-93b0-e986eb596d9c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:04.407375 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cbfce-7a1e-4c75-9e6b-9187a534bbd6 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-1a5ce1e8-e66c-47ff-93b0-e986eb596d9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:52:04.418214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:04.419367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:04.420440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:52:04.420440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:52:04.420440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:52:04.420440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:52:04.420440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:52:04.420440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:52:04.420440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:52:04.420440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:04.420440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 538/1077] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:52:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:52:04.425156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce1e8-e66c-47ff-93b0-e986eb596d9c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:04.444635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2875b0a-4b1d-455e-9e96-6c3cc362e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:04.445197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2875b0a-4b1d-455e-9e96-6c3cc362e33a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.445476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2875b0a-4b1d-455e-9e96-6c3cc362e33a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.445748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2875b0a-4b1d-455e-9e96-6c3cc362e33a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.447625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:04.456667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2875b0a-4b1d-455e-9e96-6c3cc362e33a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-a2875b0a-4b1d-455e-9e96-6c3cc362e33a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:04.461374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:04.462464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:04.463642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 540/1078] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:52:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:52:04.468336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2875b0a-4b1d-455e-9e96-6c3cc362e33a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:04.528645 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce1e8-e66c-47ff-93b0-e986eb596d9c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:04.538247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:04.564884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b334-11c0-4d18-8990-029b79c738fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:04.565288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2875b0a-4b1d-455e-9e96-6c3cc362e33a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:04.565481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b334-11c0-4d18-8990-029b79c738fa None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.565761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b334-11c0-4d18-8990-029b79c738fa None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.566036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b334-11c0-4d18-8990-029b79c738fa None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.566723 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8b334-11c0-4d18-8990-029b79c738fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:04.567053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8b334-11c0-4d18-8990-029b79c738fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:04.568479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8b334-11c0-4d18-8990-029b79c738fa None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:04.580035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 539/1079] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:52:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:04.586348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b334-11c0-4d18-8990-029b79c738fa None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:04.598484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f175385-294c-4349-8b14-3540539b391b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:04.599123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f175385-294c-4349-8b14-3540539b391b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.599407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f175385-294c-4349-8b14-3540539b391b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.599683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f175385-294c-4349-8b14-3540539b391b None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.600959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f175385-294c-4349-8b14-3540539b391b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:04.601265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f175385-294c-4349-8b14-3540539b391b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:04.602634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f175385-294c-4349-8b14-3540539b391b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:04.610476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 541/1080] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:52:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:04.616170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f175385-294c-4349-8b14-3540539b391b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:04.671675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:04.683273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b334-11c0-4d18-8990-029b79c738fa None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:04.716453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce26386-0af1-45fd-8096-ed4af2d51758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:04.717021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce26386-0af1-45fd-8096-ed4af2d51758 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.717293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce26386-0af1-45fd-8096-ed4af2d51758 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.717563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce26386-0af1-45fd-8096-ed4af2d51758 None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/904f7f036565419b9fdecc5e47acb45c/roles/b0c99aa3196841039c3bb72b951c33fb` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.718364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f175385-294c-4349-8b14-3540539b391b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:04.719712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce26386-0af1-45fd-8096-ed4af2d51758 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=904f7f036565419b9fdecc5e47acb45c, role_id=b0c99aa3196841039c3bb72b951c33fb)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:04.721122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce26386-0af1-45fd-8096-ed4af2d51758 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:04.734470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 540/1081] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:52:04 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/904f7f036565419b9fdecc5e47acb45c/roles/b0c99aa3196841039c3bb72b951c33fb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:52:04.752876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61d959-d49f-47a6-84d7-04bfe1176c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:04.753520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61d959-d49f-47a6-84d7-04bfe1176c64 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.753795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61d959-d49f-47a6-84d7-04bfe1176c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.754060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61d959-d49f-47a6-84d7-04bfe1176c64 None admin] PATH_INFO: `/v3/users/45004a2c55584f3b9686cdbdb166fc83` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.755264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61d959-d49f-47a6-84d7-04bfe1176c64 None admin] RBAC: Authorizing `identity:get_user(user_id=45004a2c55584f3b9686cdbdb166fc83)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:04.756786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61d959-d49f-47a6-84d7-04bfe1176c64 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:04.758498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 542/1082] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:04 2022] GET /identity/v3/users/45004a2c55584f3b9686cdbdb166fc83 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:04.764501 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce26386-0af1-45fd-8096-ed4af2d51758 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:04.794575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:04.867603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce26386-0af1-45fd-8096-ed4af2d51758 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:04.904872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cd998-ec0e-4122-a063-f5db5ac5d1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:04.905503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cd998-ec0e-4122-a063-f5db5ac5d1ce None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:04.905775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cd998-ec0e-4122-a063-f5db5ac5d1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:04.906041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cd998-ec0e-4122-a063-f5db5ac5d1ce None admin] PATH_INFO: `/v3/projects/35f4047759834ed49d1e59318821ad1e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:04.907194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2cd998-ec0e-4122-a063-f5db5ac5d1ce None admin] RBAC: Authorizing `identity:get_project(project_id=35f4047759834ed49d1e59318821ad1e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:04.908635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2cd998-ec0e-4122-a063-f5db5ac5d1ce None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:04.910239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 541/1083] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:04 2022] GET /identity/v3/projects/35f4047759834ed49d1e59318821ad1e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:04.916680 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61d959-d49f-47a6-84d7-04bfe1176c64 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:05.016062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61d959-d49f-47a6-84d7-04bfe1176c64 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:05.038770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:05.050092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4217c58-702a-47a4-9a55-379bac8d473e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:05.050750 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4217c58-702a-47a4-9a55-379bac8d473e None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:05.051124 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4217c58-702a-47a4-9a55-379bac8d473e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:05.051403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4217c58-702a-47a4-9a55-379bac8d473e None admin] PATH_INFO: `/v3/projects/35f4047759834ed49d1e59318821ad1e/users/45004a2c55584f3b9686cdbdb166fc83/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:05.053319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4217c58-702a-47a4-9a55-379bac8d473e None admin] RBAC: Authorizing `identity:create_grant(project_id=35f4047759834ed49d1e59318821ad1e, user_id=45004a2c55584f3b9686cdbdb166fc83, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:05.054791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4217c58-702a-47a4-9a55-379bac8d473e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:05.067064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 543/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:52:04 2022] PUT /identity/v3/projects/35f4047759834ed49d1e59318821ad1e/users/45004a2c55584f3b9686cdbdb166fc83/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:52:06.716659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272a32-c0a9-435f-9322-76a512419728 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:06.717153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272a32-c0a9-435f-9322-76a512419728 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:06.717153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272a32-c0a9-435f-9322-76a512419728 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:06.718567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 542/1085] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:06.723605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8100b-a9ea-40b3-9dd2-52aa86c298a7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:06.723982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8100b-a9ea-40b3-9dd2-52aa86c298a7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:06.724285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8100b-a9ea-40b3-9dd2-52aa86c298a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:06.741873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f9c8100b-a9ea-40b3-9dd2-52aa86c298a7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:06.743089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f9c8100b-a9ea-40b3-9dd2-52aa86c298a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:06.807984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 544/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:06.831349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272a32-c0a9-435f-9322-76a512419728 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:06.921545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:06.929148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272a32-c0a9-435f-9322-76a512419728 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:06.961583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d9952-2c5d-4242-8dd0-bb0f0e5efedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:06.962200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d9952-2c5d-4242-8dd0-bb0f0e5efedf None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:06.962474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d9952-2c5d-4242-8dd0-bb0f0e5efedf None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:06.962740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d9952-2c5d-4242-8dd0-bb0f0e5efedf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:06.972470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d9952-2c5d-4242-8dd0-bb0f0e5efedf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-2e4d9952-2c5d-4242-8dd0-bb0f0e5efedf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:06.976989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:06.978081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:06.978935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 543/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 10 09:52:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:06.984060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8100b-a9ea-40b3-9dd2-52aa86c298a7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.025288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a10ad1-ea9f-4ec6-8d0e-f1cc980f7082 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.025570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a10ad1-ea9f-4ec6-8d0e-f1cc980f7082 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.025856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a10ad1-ea9f-4ec6-8d0e-f1cc980f7082 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.027266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 544/1088] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:52:07.032194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6301479-b8a7-4197-a9c4-e96ca5128595 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.032477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6301479-b8a7-4197-a9c4-e96ca5128595 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.032751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6301479-b8a7-4197-a9c4-e96ca5128595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.050202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d6301479-b8a7-4197-a9c4-e96ca5128595 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:07.051409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d6301479-b8a7-4197-a9c4-e96ca5128595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:07.081535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8100b-a9ea-40b3-9dd2-52aa86c298a7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.095509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:07.116278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae84b4-feb4-4433-9164-cdcd0959d28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.116873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae84b4-feb4-4433-9164-cdcd0959d28c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.117088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 545/1089] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:52:07.117212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae84b4-feb4-4433-9164-cdcd0959d28c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.117402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae84b4-feb4-4433-9164-cdcd0959d28c None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.118057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae84b4-feb4-4433-9164-cdcd0959d28c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:07.118356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae84b4-feb4-4433-9164-cdcd0959d28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:07.119999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae84b4-feb4-4433-9164-cdcd0959d28c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:07.127678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 545/1090] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 10 09:52:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:07.133982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6301479-b8a7-4197-a9c4-e96ca5128595 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.142254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae84b4-feb4-4433-9164-cdcd0959d28c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.216639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:07.232047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6301479-b8a7-4197-a9c4-e96ca5128595 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.240459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae84b4-feb4-4433-9164-cdcd0959d28c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.269847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4240abe-d2db-4e27-8dae-1fcd0b3cdf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.270466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4240abe-d2db-4e27-8dae-1fcd0b3cdf63 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.270745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4240abe-d2db-4e27-8dae-1fcd0b3cdf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.271027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4240abe-d2db-4e27-8dae-1fcd0b3cdf63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.274065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9a40f-31b2-4e17-b839-615519b0a97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.274685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9a40f-31b2-4e17-b839-615519b0a97e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.274987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9a40f-31b2-4e17-b839-615519b0a97e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.275264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9a40f-31b2-4e17-b839-615519b0a97e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.281224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4240abe-d2db-4e27-8dae-1fcd0b3cdf63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:07.282650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4240abe-d2db-4e27-8dae-1fcd0b3cdf63 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:07.285590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9a40f-31b2-4e17-b839-615519b0a97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-86f9a40f-31b2-4e17-b839-615519b0a97e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:07.290718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:07.291817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:07.292824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 546/1091] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:52:07.297759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9a40f-31b2-4e17-b839-615519b0a97e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-e4240abe-d2db-4e27-8dae-1fcd0b3cdf63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:52:07.299693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:07.300808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:52:07.301862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:52:07.301862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:07.301862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:07.301862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 546/1092] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:07.306722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4240abe-d2db-4e27-8dae-1fcd0b3cdf63 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.335485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:07.398912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9a40f-31b2-4e17-b839-615519b0a97e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.404016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4240abe-d2db-4e27-8dae-1fcd0b3cdf63 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.436194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbbf00-fc2d-47fc-9c4a-673fdd5a3f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.436797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbbf00-fc2d-47fc-9c4a-673fdd5a3f17 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.437072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbbf00-fc2d-47fc-9c4a-673fdd5a3f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.437341 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbbf00-fc2d-47fc-9c4a-673fdd5a3f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.438020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbbf00-fc2d-47fc-9c4a-673fdd5a3f17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:07.438153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b2127-0d93-4817-a1aa-b6a1a59425eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.438315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbbf00-fc2d-47fc-9c4a-673fdd5a3f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:07.438695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b2127-0d93-4817-a1aa-b6a1a59425eb None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.438996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b2127-0d93-4817-a1aa-b6a1a59425eb None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.439271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b2127-0d93-4817-a1aa-b6a1a59425eb None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.439681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbbf00-fc2d-47fc-9c4a-673fdd5a3f17 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:07.439904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b2127-0d93-4817-a1aa-b6a1a59425eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:07.440203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b2127-0d93-4817-a1aa-b6a1a59425eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:07.441548 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b2127-0d93-4817-a1aa-b6a1a59425eb None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:07.448260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:07.448386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 547/1093] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:07.457822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 547/1094] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:07.457981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbbf00-fc2d-47fc-9c4a-673fdd5a3f17 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.459075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b2127-0d93-4817-a1aa-b6a1a59425eb None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.547316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:07.551587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbbf00-fc2d-47fc-9c4a-673fdd5a3f17 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.555923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b2127-0d93-4817-a1aa-b6a1a59425eb None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.585808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a14b9d-978c-4e3b-bcdd-bccdd5adb5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.586422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a14b9d-978c-4e3b-bcdd-bccdd5adb5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.586696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a14b9d-978c-4e3b-bcdd-bccdd5adb5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.586982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a14b9d-978c-4e3b-bcdd-bccdd5adb5c0 None admin] PATH_INFO: `/v3/users/45004a2c55584f3b9686cdbdb166fc83` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.588074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a14b9d-978c-4e3b-bcdd-bccdd5adb5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=45004a2c55584f3b9686cdbdb166fc83)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:07.588861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915df7f-2a8a-496b-ab3e-344fc1821672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.589476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915df7f-2a8a-496b-ab3e-344fc1821672 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.589668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a14b9d-978c-4e3b-bcdd-bccdd5adb5c0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:07.589756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915df7f-2a8a-496b-ab3e-344fc1821672 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.590011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915df7f-2a8a-496b-ab3e-344fc1821672 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.591042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 548/1095] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/users/45004a2c55584f3b9686cdbdb166fc83 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:07.596596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a14b9d-978c-4e3b-bcdd-bccdd5adb5c0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.597176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915df7f-2a8a-496b-ab3e-344fc1821672 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:07.598589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915df7f-2a8a-496b-ab3e-344fc1821672 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-c915df7f-2a8a-496b-ab3e-344fc1821672 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:52:07.610479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:07.611466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 548/1096] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:52:07.617124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915df7f-2a8a-496b-ab3e-344fc1821672 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.673394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:07.693059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a14b9d-978c-4e3b-bcdd-bccdd5adb5c0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.713924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915df7f-2a8a-496b-ab3e-344fc1821672 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.726207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499d754-89a3-4d10-8b09-b6ccc168bde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.726802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d754-89a3-4d10-8b09-b6ccc168bde8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.727104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d754-89a3-4d10-8b09-b6ccc168bde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.727373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d754-89a3-4d10-8b09-b6ccc168bde8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.746895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74a214-e1ab-48e9-abc0-780db5c351e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.747518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74a214-e1ab-48e9-abc0-780db5c351e8 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.747798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74a214-e1ab-48e9-abc0-780db5c351e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.748067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74a214-e1ab-48e9-abc0-780db5c351e8 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.748782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74a214-e1ab-48e9-abc0-780db5c351e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:07.749086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74a214-e1ab-48e9-abc0-780db5c351e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:07.750495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74a214-e1ab-48e9-abc0-780db5c351e8 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:07.759882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:07.766865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 549/1097] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:07.772923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74a214-e1ab-48e9-abc0-780db5c351e8 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.794550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499d754-89a3-4d10-8b09-b6ccc168bde8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:07.796027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499d754-89a3-4d10-8b09-b6ccc168bde8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:07.867885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74a214-e1ab-48e9-abc0-780db5c351e8 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:07.883951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:07.890968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 549/1098] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 09:52:07.897402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499d754-89a3-4d10-8b09-b6ccc168bde8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:07.906685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cb572-45bf-44d9-a85e-2c438dd7fed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:07.907315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cb572-45bf-44d9-a85e-2c438dd7fed4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:07.907594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cb572-45bf-44d9-a85e-2c438dd7fed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:07.907867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cb572-45bf-44d9-a85e-2c438dd7fed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:07.959131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:07.973708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cb572-45bf-44d9-a85e-2c438dd7fed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:07.975206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cb572-45bf-44d9-a85e-2c438dd7fed4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:07.992692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499d754-89a3-4d10-8b09-b6ccc168bde8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:08.025667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df52c48-4988-42d0-8fcd-48e1e1bf3175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:08.026277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df52c48-4988-42d0-8fcd-48e1e1bf3175 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:08.026548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df52c48-4988-42d0-8fcd-48e1e1bf3175 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:08.026812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df52c48-4988-42d0-8fcd-48e1e1bf3175 None admin] PATH_INFO: `/v3/projects/35f4047759834ed49d1e59318821ad1e` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:08.027931 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df52c48-4988-42d0-8fcd-48e1e1bf3175 None admin] RBAC: Authorizing `identity:get_project(project_id=35f4047759834ed49d1e59318821ad1e)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:08.029314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df52c48-4988-42d0-8fcd-48e1e1bf3175 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:08.031195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 550/1099] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/projects/35f4047759834ed49d1e59318821ad1e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:08.036987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df52c48-4988-42d0-8fcd-48e1e1bf3175 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:08.071707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 550/1100] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 09:52:08.077097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cb572-45bf-44d9-a85e-2c438dd7fed4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:08.093538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:08.133496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df52c48-4988-42d0-8fcd-48e1e1bf3175 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:08.166048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009acc-0e0d-4a89-b4c9-1c16658a7008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:08.166640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009acc-0e0d-4a89-b4c9-1c16658a7008 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:08.166916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009acc-0e0d-4a89-b4c9-1c16658a7008 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:08.167215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009acc-0e0d-4a89-b4c9-1c16658a7008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:08.167929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06009acc-0e0d-4a89-b4c9-1c16658a7008 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=35f4047759834ed49d1e59318821ad1e, user.id=45004a2c55584f3b9686cdbdb166fc83) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:08.168231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06009acc-0e0d-4a89-b4c9-1c16658a7008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:08.169647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06009acc-0e0d-4a89-b4c9-1c16658a7008 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:08.171363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:08.172481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cb572-45bf-44d9-a85e-2c438dd7fed4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:08.176407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 551/1101] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:08 2022] GET /identity/v3/role_assignments?user.id=45004a2c55584f3b9686cdbdb166fc83&scope.project.id=35f4047759834ed49d1e59318821ad1e&role.id=8417343e9ef64991bcf51bb9692da102 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:08.216651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c9ead-2a8a-4895-810f-781380314329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:08.217283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c9ead-2a8a-4895-810f-781380314329 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:08.217554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c9ead-2a8a-4895-810f-781380314329 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:08.217822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c9ead-2a8a-4895-810f-781380314329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:08.228840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c9ead-2a8a-4895-810f-781380314329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:08.230314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c9ead-2a8a-4895-810f-781380314329 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-a07c9ead-2a8a-4895-810f-781380314329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:52:08.246319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:08.247403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:52:08.248281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:52:08.248281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:52:08.248281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:08.251118 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 551/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:52:08.253244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009acc-0e0d-4a89-b4c9-1c16658a7008 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:08.355248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009acc-0e0d-4a89-b4c9-1c16658a7008 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:08.390006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b750ba-1d1d-4c6c-b5a0-5f42d7c4cd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:08.390519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b750ba-1d1d-4c6c-b5a0-5f42d7c4cd3e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:08.390799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b750ba-1d1d-4c6c-b5a0-5f42d7c4cd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:08.391104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b750ba-1d1d-4c6c-b5a0-5f42d7c4cd3e None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:08.391747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b750ba-1d1d-4c6c-b5a0-5f42d7c4cd3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:08.392052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b750ba-1d1d-4c6c-b5a0-5f42d7c4cd3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:08.393434 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b750ba-1d1d-4c6c-b5a0-5f42d7c4cd3e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:08.405866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 552/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:08.411665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c9ead-2a8a-4895-810f-781380314329 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:08.476897 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:08.511571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c9ead-2a8a-4895-810f-781380314329 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:08.544497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346059e-56c2-4c7f-8345-d2f9faeb33b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:08.545122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346059e-56c2-4c7f-8345-d2f9faeb33b6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:08.545401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346059e-56c2-4c7f-8345-d2f9faeb33b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:08.545674 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346059e-56c2-4c7f-8345-d2f9faeb33b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:08.552246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346059e-56c2-4c7f-8345-d2f9faeb33b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:08.554196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346059e-56c2-4c7f-8345-d2f9faeb33b6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-6346059e-56c2-4c7f-8345-d2f9faeb33b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:52:08.565019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:08.565864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:08.566752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:52:08.566752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:52:08.566752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:52:08.566752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:52:08.566752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:52:08.566752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:52:08.566752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:52:08.566752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:08.566752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 552/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:52:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:08.572589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b750ba-1d1d-4c6c-b5a0-5f42d7c4cd3e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:08.628772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:08.670728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b750ba-1d1d-4c6c-b5a0-5f42d7c4cd3e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:08.703714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a52172-b082-41ba-8326-57268faa0c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:08.704310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a52172-b082-41ba-8326-57268faa0c76 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:08.704584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a52172-b082-41ba-8326-57268faa0c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:08.704852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a52172-b082-41ba-8326-57268faa0c76 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:08.705523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a52172-b082-41ba-8326-57268faa0c76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:08.705820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a52172-b082-41ba-8326-57268faa0c76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:08.707258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a52172-b082-41ba-8326-57268faa0c76 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:08.718206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 553/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:08.724496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346059e-56c2-4c7f-8345-d2f9faeb33b6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:08.820704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346059e-56c2-4c7f-8345-d2f9faeb33b6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:08.845422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:08.853898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7a9e9-bf78-43d4-8e44-f7a752733f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:08.854500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7a9e9-bf78-43d4-8e44-f7a752733f23 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:08.854776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7a9e9-bf78-43d4-8e44-f7a752733f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:08.855074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7a9e9-bf78-43d4-8e44-f7a752733f23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:08.855794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7a9e9-bf78-43d4-8e44-f7a752733f23 None admin] RBAC: Adding query filter params (role.id=b0c99aa3196841039c3bb72b951c33fb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=904f7f036565419b9fdecc5e47acb45c) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:08.856107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7a9e9-bf78-43d4-8e44-f7a752733f23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:08.857523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7a9e9-bf78-43d4-8e44-f7a752733f23 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:08.863104 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 553/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:08 2022] GET /identity/v3/role_assignments?user.id=904f7f036565419b9fdecc5e47acb45c&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=b0c99aa3196841039c3bb72b951c33fb => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:10.195892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4cc33-779a-4941-822a-175c6e35a1a1 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:10.196456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4cc33-779a-4941-822a-175c6e35a1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:10.196456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4cc33-779a-4941-822a-175c6e35a1a1 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:10.197759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 554/1107] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:52:10.202818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5bf1-378c-4cc0-bf9f-15ee47c698bf None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:10.203275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5bf1-378c-4cc0-bf9f-15ee47c698bf None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:10.203396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5bf1-378c-4cc0-bf9f-15ee47c698bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:10.220907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-834b5bf1-378c-4cc0-bf9f-15ee47c698bf None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:10.222069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-834b5bf1-378c-4cc0-bf9f-15ee47c698bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:10.286678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 554/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:10.311297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4cc33-779a-4941-822a-175c6e35a1a1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:10.408897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4cc33-779a-4941-822a-175c6e35a1a1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:10.426081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:10.442603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f386f-cec4-421c-8170-21a33618e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:10.443246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f386f-cec4-421c-8170-21a33618e67a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:10.443520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f386f-cec4-421c-8170-21a33618e67a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:10.443783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f386f-cec4-421c-8170-21a33618e67a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:10.444857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f386f-cec4-421c-8170-21a33618e67a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:10.446308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f386f-cec4-421c-8170-21a33618e67a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:10.447832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 555/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:52:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:10.453520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b5bf1-378c-4cc0-bf9f-15ee47c698bf None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:10.546755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b5bf1-378c-4cc0-bf9f-15ee47c698bf None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:10.569240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:10.580065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4479fc9-0d8b-4664-94af-15f735a294c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:10.580653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4479fc9-0d8b-4664-94af-15f735a294c8 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:10.580928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4479fc9-0d8b-4664-94af-15f735a294c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:10.581195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4479fc9-0d8b-4664-94af-15f735a294c8 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:10.581831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4479fc9-0d8b-4664-94af-15f735a294c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:10.583268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4479fc9-0d8b-4664-94af-15f735a294c8 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:10.604861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 555/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:52:10 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:52:12.612149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da1602-1a16-4fde-958b-77fd1552fa66 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:12.612654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da1602-1a16-4fde-958b-77fd1552fa66 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:12.612744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da1602-1a16-4fde-958b-77fd1552fa66 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:12.614006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 556/1111] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:12.619477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e466c9-bbb1-4a06-8b19-044263828166 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:12.619910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e466c9-bbb1-4a06-8b19-044263828166 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:12.620025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e466c9-bbb1-4a06-8b19-044263828166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:12.637909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-68e466c9-bbb1-4a06-8b19-044263828166 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:12.639120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-68e466c9-bbb1-4a06-8b19-044263828166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:12.701788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 556/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:12.725081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da1602-1a16-4fde-958b-77fd1552fa66 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:12.820886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da1602-1a16-4fde-958b-77fd1552fa66 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:12.852157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39703b8-90d5-4772-9ccd-269c26066d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:12.852756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39703b8-90d5-4772-9ccd-269c26066d1d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:12.853031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39703b8-90d5-4772-9ccd-269c26066d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:12.853299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39703b8-90d5-4772-9ccd-269c26066d1d None admin] PATH_INFO: `/v3/roles/02c7974bceec49ee8895dabf30cc8fff` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:12.854655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39703b8-90d5-4772-9ccd-269c26066d1d None admin] RBAC: Authorizing `identity:get_role(role_id=02c7974bceec49ee8895dabf30cc8fff)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:12.856017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39703b8-90d5-4772-9ccd-269c26066d1d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:12.857277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 557/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:12 2022] GET /identity/v3/roles/02c7974bceec49ee8895dabf30cc8fff => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:12.862748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e466c9-bbb1-4a06-8b19-044263828166 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:12.956933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e466c9-bbb1-4a06-8b19-044263828166 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:12.965604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:12.990159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda256e-4dc4-4913-bfb4-9302a7c7dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:12.991321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda256e-4dc4-4913-bfb4-9302a7c7dca6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:12.991594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda256e-4dc4-4913-bfb4-9302a7c7dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:12.991863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda256e-4dc4-4913-bfb4-9302a7c7dca6 None admin] PATH_INFO: `/v3/users/cf41f856295e43e39db4c858a450ec5c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:13.004392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda256e-4dc4-4913-bfb4-9302a7c7dca6 None admin] RBAC: Authorizing `identity:get_user(user_id=cf41f856295e43e39db4c858a450ec5c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:13.005918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda256e-4dc4-4913-bfb4-9302a7c7dca6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:13.008113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 557/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:12 2022] GET /identity/v3/users/cf41f856295e43e39db4c858a450ec5c => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:13.013867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39703b8-90d5-4772-9ccd-269c26066d1d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:13.065764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:13.108292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39703b8-90d5-4772-9ccd-269c26066d1d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:13.143675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c178b7-37a0-442b-b632-08207521931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:13.144297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c178b7-37a0-442b-b632-08207521931c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:13.144571 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c178b7-37a0-442b-b632-08207521931c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:13.144841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c178b7-37a0-442b-b632-08207521931c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:13.209330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c178b7-37a0-442b-b632-08207521931c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:13.210702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c178b7-37a0-442b-b632-08207521931c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:13.261622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:13.306885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 558/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:52:13.312795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda256e-4dc4-4913-bfb4-9302a7c7dca6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:13.407216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda256e-4dc4-4913-bfb4-9302a7c7dca6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:13.440570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4689475-e1bf-43b5-81ae-1bbf3dc29e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:13.441172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4689475-e1bf-43b5-81ae-1bbf3dc29e04 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:13.441446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4689475-e1bf-43b5-81ae-1bbf3dc29e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:13.441711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4689475-e1bf-43b5-81ae-1bbf3dc29e04 None admin] PATH_INFO: `/v3/projects/35f4047759834ed49d1e59318821ad1e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:13.442787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4689475-e1bf-43b5-81ae-1bbf3dc29e04 None admin] RBAC: Authorizing `identity:get_project(project_id=35f4047759834ed49d1e59318821ad1e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:13.444256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4689475-e1bf-43b5-81ae-1bbf3dc29e04 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:13.445830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 558/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:13 2022] GET /identity/v3/projects/35f4047759834ed49d1e59318821ad1e => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:13.451850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c178b7-37a0-442b-b632-08207521931c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:13.546747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c178b7-37a0-442b-b632-08207521931c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:13.578705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9ba81-7caf-47f0-9274-05df148a3f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:13.579315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9ba81-7caf-47f0-9274-05df148a3f8f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:13.579591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9ba81-7caf-47f0-9274-05df148a3f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:13.579857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9ba81-7caf-47f0-9274-05df148a3f8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:13.580559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9ba81-7caf-47f0-9274-05df148a3f8f None admin] RBAC: Adding query filter params (role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=35f4047759834ed49d1e59318821ad1e, user.id=cf41f856295e43e39db4c858a450ec5c) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:13.580858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9ba81-7caf-47f0-9274-05df148a3f8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:13.582245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9ba81-7caf-47f0-9274-05df148a3f8f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:13.587850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 559/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:13 2022] GET /identity/v3/role_assignments?user.id=cf41f856295e43e39db4c858a450ec5c&scope.project.id=35f4047759834ed49d1e59318821ad1e&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:15.530851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9806590-0ba7-48cd-9605-bf364ba707ae None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:15.531498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9806590-0ba7-48cd-9605-bf364ba707ae None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:15.531498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9806590-0ba7-48cd-9605-bf364ba707ae None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:15.532714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 559/1118] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:15.538206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daceb274-863f-48e8-9b20-629578ecc8e5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:15.538639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daceb274-863f-48e8-9b20-629578ecc8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:15.538737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daceb274-863f-48e8-9b20-629578ecc8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:15.556791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-daceb274-863f-48e8-9b20-629578ecc8e5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:15.557978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-daceb274-863f-48e8-9b20-629578ecc8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:15.570558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:15.622913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 560/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:15.644639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9806590-0ba7-48cd-9605-bf364ba707ae None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:15.678842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:15.741940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9806590-0ba7-48cd-9605-bf364ba707ae None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:15.773694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7b9d7-d513-4b28-ad42-fa31c1228672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:15.774294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7b9d7-d513-4b28-ad42-fa31c1228672 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:15.774566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7b9d7-d513-4b28-ad42-fa31c1228672 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:15.774831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7b9d7-d513-4b28-ad42-fa31c1228672 None admin] PATH_INFO: `/v3/roles/02c7974bceec49ee8895dabf30cc8fff` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:15.776188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7b9d7-d513-4b28-ad42-fa31c1228672 None admin] RBAC: Authorizing `identity:get_role(role_id=02c7974bceec49ee8895dabf30cc8fff)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:15.777535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7b9d7-d513-4b28-ad42-fa31c1228672 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:15.778797 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 560/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:15 2022] GET /identity/v3/roles/02c7974bceec49ee8895dabf30cc8fff => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:15.784335 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daceb274-863f-48e8-9b20-629578ecc8e5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:15.878150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daceb274-863f-48e8-9b20-629578ecc8e5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:15.910100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5360d-2da7-4c5a-a79e-0f0f8c852eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:15.910712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5360d-2da7-4c5a-a79e-0f0f8c852eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:15.911019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5360d-2da7-4c5a-a79e-0f0f8c852eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:15.911297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5360d-2da7-4c5a-a79e-0f0f8c852eb5 None admin] PATH_INFO: `/v3/users/cf41f856295e43e39db4c858a450ec5c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:15.913039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5360d-2da7-4c5a-a79e-0f0f8c852eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=cf41f856295e43e39db4c858a450ec5c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:15.914434 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5360d-2da7-4c5a-a79e-0f0f8c852eb5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:15.915993 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 561/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:15 2022] GET /identity/v3/users/cf41f856295e43e39db4c858a450ec5c => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:15.921420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7b9d7-d513-4b28-ad42-fa31c1228672 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:16.019683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7b9d7-d513-4b28-ad42-fa31c1228672 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:16.021606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:16.055259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5578-c516-410f-986c-fadd44806de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:16.055866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5578-c516-410f-986c-fadd44806de9 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:16.056139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5578-c516-410f-986c-fadd44806de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:16.056409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5578-c516-410f-986c-fadd44806de9 None admin] PATH_INFO: `/v3/projects/35f4047759834ed49d1e59318821ad1e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:16.057506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c5578-c516-410f-986c-fadd44806de9 None admin] RBAC: Authorizing `identity:get_project(project_id=35f4047759834ed49d1e59318821ad1e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:16.058923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c5578-c516-410f-986c-fadd44806de9 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:16.060535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 561/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:15 2022] GET /identity/v3/projects/35f4047759834ed49d1e59318821ad1e => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:16.066511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5360d-2da7-4c5a-a79e-0f0f8c852eb5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:16.084840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:16.162911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5360d-2da7-4c5a-a79e-0f0f8c852eb5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:16.194869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afb654-8549-475c-83f9-ff010a482646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:16.195533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afb654-8549-475c-83f9-ff010a482646 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:16.195809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afb654-8549-475c-83f9-ff010a482646 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:16.196079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afb654-8549-475c-83f9-ff010a482646 None admin] PATH_INFO: `/v3/projects/35f4047759834ed49d1e59318821ad1e/users/cf41f856295e43e39db4c858a450ec5c/roles/02c7974bceec49ee8895dabf30cc8fff` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:16.198151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afb654-8549-475c-83f9-ff010a482646 None admin] RBAC: Authorizing `identity:create_grant(project_id=35f4047759834ed49d1e59318821ad1e, user_id=cf41f856295e43e39db4c858a450ec5c, role_id=02c7974bceec49ee8895dabf30cc8fff)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:16.199596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afb654-8549-475c-83f9-ff010a482646 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:16.211004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 562/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:52:16 2022] PUT /identity/v3/projects/35f4047759834ed49d1e59318821ad1e/users/cf41f856295e43e39db4c858a450ec5c/roles/02c7974bceec49ee8895dabf30cc8fff => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:52:18.172188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae47ac-5822-4533-afe0-ba71ccabcbc0 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:18.172736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae47ac-5822-4533-afe0-ba71ccabcbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:18.172736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae47ac-5822-4533-afe0-ba71ccabcbc0 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:18.174062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 562/1124] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:18.179868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f808f-c4f7-4556-bc57-c0c7c0d8db50 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:18.180286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f808f-c4f7-4556-bc57-c0c7c0d8db50 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:18.180412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f808f-c4f7-4556-bc57-c0c7c0d8db50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:18.198287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-509f808f-c4f7-4556-bc57-c0c7c0d8db50 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:18.199482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-509f808f-c4f7-4556-bc57-c0c7c0d8db50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:18.263069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 563/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:52:18.285463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae47ac-5822-4533-afe0-ba71ccabcbc0 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:18.384292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae47ac-5822-4533-afe0-ba71ccabcbc0 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:18.416270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe32026-0fee-4ea3-8938-f23cfd6dbd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:18.416887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe32026-0fee-4ea3-8938-f23cfd6dbd15 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:18.417164 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe32026-0fee-4ea3-8938-f23cfd6dbd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:18.417434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe32026-0fee-4ea3-8938-f23cfd6dbd15 None admin] PATH_INFO: `/v3/roles/02c7974bceec49ee8895dabf30cc8fff` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:18.418803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe32026-0fee-4ea3-8938-f23cfd6dbd15 None admin] RBAC: Authorizing `identity:get_role(role_id=02c7974bceec49ee8895dabf30cc8fff)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:18.420195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe32026-0fee-4ea3-8938-f23cfd6dbd15 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:18.421507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 563/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:18 2022] GET /identity/v3/roles/02c7974bceec49ee8895dabf30cc8fff => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:18.427397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f808f-c4f7-4556-bc57-c0c7c0d8db50 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:18.436087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:18.524878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f808f-c4f7-4556-bc57-c0c7c0d8db50 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:18.558100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346ca1c-632a-4a6a-90e7-3270f937c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:18.558720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346ca1c-632a-4a6a-90e7-3270f937c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:18.559025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346ca1c-632a-4a6a-90e7-3270f937c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:18.559300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346ca1c-632a-4a6a-90e7-3270f937c3d9 None admin] PATH_INFO: `/v3/users/cf41f856295e43e39db4c858a450ec5c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:18.560420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346ca1c-632a-4a6a-90e7-3270f937c3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=cf41f856295e43e39db4c858a450ec5c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:18.561942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346ca1c-632a-4a6a-90e7-3270f937c3d9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:18.563555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 564/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:18 2022] GET /identity/v3/users/cf41f856295e43e39db4c858a450ec5c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:18.569381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe32026-0fee-4ea3-8938-f23cfd6dbd15 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:18.607736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:18.665605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe32026-0fee-4ea3-8938-f23cfd6dbd15 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:18.697938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f693146-73ff-4783-866d-1751b8bc5cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:18.698472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f693146-73ff-4783-866d-1751b8bc5cdf None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:18.698751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f693146-73ff-4783-866d-1751b8bc5cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:18.699049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f693146-73ff-4783-866d-1751b8bc5cdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:18.762705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f693146-73ff-4783-866d-1751b8bc5cdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:18.764173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f693146-73ff-4783-866d-1751b8bc5cdf None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:18.799874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:18.858347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 564/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 09:52:18.864096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346ca1c-632a-4a6a-90e7-3270f937c3d9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:18.935380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:18.958672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346ca1c-632a-4a6a-90e7-3270f937c3d9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:18.990535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928aba2b-fef3-4338-96f4-d1af4a90dfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:18.991165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928aba2b-fef3-4338-96f4-d1af4a90dfb0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:18.991441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928aba2b-fef3-4338-96f4-d1af4a90dfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:18.991706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928aba2b-fef3-4338-96f4-d1af4a90dfb0 None admin] PATH_INFO: `/v3/projects/35f4047759834ed49d1e59318821ad1e` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:18.992799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928aba2b-fef3-4338-96f4-d1af4a90dfb0 None admin] RBAC: Authorizing `identity:get_project(project_id=35f4047759834ed49d1e59318821ad1e)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:18.994457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928aba2b-fef3-4338-96f4-d1af4a90dfb0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:18.996060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 565/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:18 2022] GET /identity/v3/projects/35f4047759834ed49d1e59318821ad1e => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:19.001899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f693146-73ff-4783-866d-1751b8bc5cdf None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:19.096777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f693146-73ff-4783-866d-1751b8bc5cdf None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:19.129781 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336613c-4d77-4aad-a802-745336949ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:19.130369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336613c-4d77-4aad-a802-745336949ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:19.130647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336613c-4d77-4aad-a802-745336949ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:19.130939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336613c-4d77-4aad-a802-745336949ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:19.131677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336613c-4d77-4aad-a802-745336949ce6 None admin] RBAC: Adding query filter params (role.id=02c7974bceec49ee8895dabf30cc8fff, scope.project.id=35f4047759834ed49d1e59318821ad1e, user.id=cf41f856295e43e39db4c858a450ec5c) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:19.131982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336613c-4d77-4aad-a802-745336949ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:19.133383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336613c-4d77-4aad-a802-745336949ce6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:19.138893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 565/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:19 2022] GET /identity/v3/role_assignments?user.id=cf41f856295e43e39db4c858a450ec5c&scope.project.id=35f4047759834ed49d1e59318821ad1e&role.id=02c7974bceec49ee8895dabf30cc8fff => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:21.133567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2a6a-aa8b-4434-8780-bdad481f46c5 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:21.134118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2a6a-aa8b-4434-8780-bdad481f46c5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:21.134118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2a6a-aa8b-4434-8780-bdad481f46c5 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:21.135509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 566/1131] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:21.140569 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba1ed1-e642-4114-9cef-60c1d919371c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:21.140995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba1ed1-e642-4114-9cef-60c1d919371c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:21.141135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba1ed1-e642-4114-9cef-60c1d919371c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:21.147358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:21.159572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-90ba1ed1-e642-4114-9cef-60c1d919371c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:21.160730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-90ba1ed1-e642-4114-9cef-60c1d919371c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:21.222792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 566/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:21.245833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b2a6a-aa8b-4434-8780-bdad481f46c5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:21.344969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b2a6a-aa8b-4434-8780-bdad481f46c5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:21.378172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c89b8-f2f4-4372-8039-2271abf241c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:21.378795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c89b8-f2f4-4372-8039-2271abf241c9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:21.379105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c89b8-f2f4-4372-8039-2271abf241c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:21.379377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c89b8-f2f4-4372-8039-2271abf241c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:21.380482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c89b8-f2f4-4372-8039-2271abf241c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:21.382454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c89b8-f2f4-4372-8039-2271abf241c9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:21.384098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 567/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:52:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:21.389948 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba1ed1-e642-4114-9cef-60c1d919371c None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:21.485786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba1ed1-e642-4114-9cef-60c1d919371c None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:21.514931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:21.520315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde6695-8776-457a-a5fd-2a71e5d3f320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:21.520910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde6695-8776-457a-a5fd-2a71e5d3f320 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:21.521184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde6695-8776-457a-a5fd-2a71e5d3f320 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:21.521449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde6695-8776-457a-a5fd-2a71e5d3f320 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:21.522108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde6695-8776-457a-a5fd-2a71e5d3f320 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:21.523566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde6695-8776-457a-a5fd-2a71e5d3f320 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:21.549362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 567/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:52:21 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:52:23.557559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5fa20-9888-460f-9d54-2fcd4f1e9e61 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:23.558058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5fa20-9888-460f-9d54-2fcd4f1e9e61 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:23.558174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5fa20-9888-460f-9d54-2fcd4f1e9e61 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:23.559478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 568/1135] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:23.564591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c29683-f4d7-4130-8790-8548c4bc6678 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:23.565020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c29683-f4d7-4130-8790-8548c4bc6678 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:23.565120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c29683-f4d7-4130-8790-8548c4bc6678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:23.583092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-73c29683-f4d7-4130-8790-8548c4bc6678 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:23.584244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-73c29683-f4d7-4130-8790-8548c4bc6678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:23.646619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 568/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:23.669959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5fa20-9888-460f-9d54-2fcd4f1e9e61 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:23.682305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:23.770166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5fa20-9888-460f-9d54-2fcd4f1e9e61 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:23.802470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd13b4c-7dd9-4bf7-95ac-ca7b74f328e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:23.803761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13b4c-7dd9-4bf7-95ac-ca7b74f328e9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:23.804035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13b4c-7dd9-4bf7-95ac-ca7b74f328e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:23.804302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13b4c-7dd9-4bf7-95ac-ca7b74f328e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:23.805407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd13b4c-7dd9-4bf7-95ac-ca7b74f328e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:23.806819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd13b4c-7dd9-4bf7-95ac-ca7b74f328e9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:23.808387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 569/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:52:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:23.814131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c29683-f4d7-4130-8790-8548c4bc6678 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:23.906351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c29683-f4d7-4130-8790-8548c4bc6678 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:23.919806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:23.940735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b627877-5e6f-4fb9-bf27-82e35dd50907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:23.941272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b627877-5e6f-4fb9-bf27-82e35dd50907 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:23.941551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b627877-5e6f-4fb9-bf27-82e35dd50907 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:23.941823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b627877-5e6f-4fb9-bf27-82e35dd50907 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:23.942463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b627877-5e6f-4fb9-bf27-82e35dd50907 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:23.943908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b627877-5e6f-4fb9-bf27-82e35dd50907 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:23.964152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 569/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:52:23 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:52:25.978417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87dc50-dcb9-4df5-8298-3169e43ab8d3 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:25.979231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87dc50-dcb9-4df5-8298-3169e43ab8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:25.979231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87dc50-dcb9-4df5-8298-3169e43ab8d3 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:25.980313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 570/1139] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:25.985395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2efe17-ad69-4355-a56c-5385d6739adb None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:25.985830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2efe17-ad69-4355-a56c-5385d6739adb None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:25.985929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2efe17-ad69-4355-a56c-5385d6739adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:26.003800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-4d2efe17-ad69-4355-a56c-5385d6739adb None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:26.004973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4d2efe17-ad69-4355-a56c-5385d6739adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:26.069102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 570/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:26.092043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87dc50-dcb9-4df5-8298-3169e43ab8d3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:26.167633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:26.189032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87dc50-dcb9-4df5-8298-3169e43ab8d3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:26.221305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebab954-9c37-4819-9fdf-c34b23203556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:26.221905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebab954-9c37-4819-9fdf-c34b23203556 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:26.222185 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebab954-9c37-4819-9fdf-c34b23203556 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:26.222450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebab954-9c37-4819-9fdf-c34b23203556 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:26.232025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebab954-9c37-4819-9fdf-c34b23203556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-bebab954-9c37-4819-9fdf-c34b23203556 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:26.236487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:26.237588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:26.238515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 571/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:52:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:26.243489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2efe17-ad69-4355-a56c-5385d6739adb None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:26.337706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:26.340819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2efe17-ad69-4355-a56c-5385d6739adb None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:26.374194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99249e05-c3c5-46f5-ac20-e53d416b0f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:26.374792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99249e05-c3c5-46f5-ac20-e53d416b0f85 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:26.375100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99249e05-c3c5-46f5-ac20-e53d416b0f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:26.375370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99249e05-c3c5-46f5-ac20-e53d416b0f85 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:26.376024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99249e05-c3c5-46f5-ac20-e53d416b0f85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:26.376323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99249e05-c3c5-46f5-ac20-e53d416b0f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:26.377707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99249e05-c3c5-46f5-ac20-e53d416b0f85 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:26.385986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 571/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:52:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:26.391895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebab954-9c37-4819-9fdf-c34b23203556 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:26.487786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebab954-9c37-4819-9fdf-c34b23203556 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:26.520745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b494811-575f-4a7e-aa3b-e8af9adb0ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:26.521383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b494811-575f-4a7e-aa3b-e8af9adb0ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:26.521663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b494811-575f-4a7e-aa3b-e8af9adb0ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:26.521940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b494811-575f-4a7e-aa3b-e8af9adb0ea6 None admin] PATH_INFO: `/v3/users/835f16cacf6b44ca9b2bcc6e5b0f91e1` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:26.529278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:26.535328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b494811-575f-4a7e-aa3b-e8af9adb0ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=835f16cacf6b44ca9b2bcc6e5b0f91e1)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:26.536758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b494811-575f-4a7e-aa3b-e8af9adb0ea6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:26.538365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 572/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:26 2022] GET /identity/v3/users/835f16cacf6b44ca9b2bcc6e5b0f91e1 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:26.544092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99249e05-c3c5-46f5-ac20-e53d416b0f85 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:26.639136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99249e05-c3c5-46f5-ac20-e53d416b0f85 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:26.670157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9a276-40d4-4608-b043-90c09bd9ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:26.670739 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9a276-40d4-4608-b043-90c09bd9ab05 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:26.671041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9a276-40d4-4608-b043-90c09bd9ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:26.671313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9a276-40d4-4608-b043-90c09bd9ab05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:26.699964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:26.735732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9a276-40d4-4608-b043-90c09bd9ab05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:26.737154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9a276-40d4-4608-b043-90c09bd9ab05 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:26.829880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 572/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:52:26.835855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b494811-575f-4a7e-aa3b-e8af9adb0ea6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:26.930272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b494811-575f-4a7e-aa3b-e8af9adb0ea6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:26.961550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6532b0-9475-4b97-acd5-c941eafc2dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:26.962146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6532b0-9475-4b97-acd5-c941eafc2dac None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:26.962419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6532b0-9475-4b97-acd5-c941eafc2dac None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:26.962686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6532b0-9475-4b97-acd5-c941eafc2dac None admin] PATH_INFO: `/v3/projects/47304b67d84b46e8a0278d30cffe02ff` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:26.963784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6532b0-9475-4b97-acd5-c941eafc2dac None admin] RBAC: Authorizing `identity:get_project(project_id=47304b67d84b46e8a0278d30cffe02ff)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:26.965171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6532b0-9475-4b97-acd5-c941eafc2dac None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:26.966722 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 573/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:26 2022] GET /identity/v3/projects/47304b67d84b46e8a0278d30cffe02ff => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:26.972532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9a276-40d4-4608-b043-90c09bd9ab05 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:27.030094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:27.067128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9a276-40d4-4608-b043-90c09bd9ab05 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:27.099081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9fdd8-9023-4935-8e0a-f66aa97f3d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:27.100193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9fdd8-9023-4935-8e0a-f66aa97f3d3e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:27.100468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9fdd8-9023-4935-8e0a-f66aa97f3d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:27.100733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9fdd8-9023-4935-8e0a-f66aa97f3d3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:27.101441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9fdd8-9023-4935-8e0a-f66aa97f3d3e None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=47304b67d84b46e8a0278d30cffe02ff, user.id=835f16cacf6b44ca9b2bcc6e5b0f91e1) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:27.101740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9fdd8-9023-4935-8e0a-f66aa97f3d3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:27.103158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9fdd8-9023-4935-8e0a-f66aa97f3d3e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:27.108216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 573/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:26 2022] GET /identity/v3/role_assignments?user.id=835f16cacf6b44ca9b2bcc6e5b0f91e1&scope.project.id=47304b67d84b46e8a0278d30cffe02ff&role.id=8417343e9ef64991bcf51bb9692da102 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:29.063240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11701f28-ad08-44fb-aac0-edeecffdd636 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:29.063771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11701f28-ad08-44fb-aac0-edeecffdd636 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:29.063771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11701f28-ad08-44fb-aac0-edeecffdd636 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:29.065093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 574/1147] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:29.070371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51fcd-4889-4b9e-9eab-c92df927a81d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:29.070764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51fcd-4889-4b9e-9eab-c92df927a81d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:29.070911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51fcd-4889-4b9e-9eab-c92df927a81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:29.088387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-53c51fcd-4889-4b9e-9eab-c92df927a81d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:29.089552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-53c51fcd-4889-4b9e-9eab-c92df927a81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:29.152909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 574/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:52:29.174817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11701f28-ad08-44fb-aac0-edeecffdd636 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:29.234961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:29.271961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11701f28-ad08-44fb-aac0-edeecffdd636 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:29.304403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5170d-e6db-4323-83b0-3c9edf1d0393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:29.305004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5170d-e6db-4323-83b0-3c9edf1d0393 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:29.305281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5170d-e6db-4323-83b0-3c9edf1d0393 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:29.305545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5170d-e6db-4323-83b0-3c9edf1d0393 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:29.315150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5170d-e6db-4323-83b0-3c9edf1d0393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-35e5170d-e6db-4323-83b0-3c9edf1d0393 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:29.319672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:29.320733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:29.321719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 575/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:52:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:29.326622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c51fcd-4889-4b9e-9eab-c92df927a81d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:29.385192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:29.428184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c51fcd-4889-4b9e-9eab-c92df927a81d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:29.460136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782a91c-9002-474e-8f53-6bdb860cc52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:29.460725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782a91c-9002-474e-8f53-6bdb860cc52e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:29.460997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782a91c-9002-474e-8f53-6bdb860cc52e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:29.461263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782a91c-9002-474e-8f53-6bdb860cc52e None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:29.461907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782a91c-9002-474e-8f53-6bdb860cc52e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:29.462199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782a91c-9002-474e-8f53-6bdb860cc52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:29.463580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782a91c-9002-474e-8f53-6bdb860cc52e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:29.471115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 575/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:52:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:29.477711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5170d-e6db-4323-83b0-3c9edf1d0393 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:29.571176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5170d-e6db-4323-83b0-3c9edf1d0393 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:29.593242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:29.604184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f0e9a-7199-437a-b74f-ea3c7a46be40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:29.604785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f0e9a-7199-437a-b74f-ea3c7a46be40 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:29.605058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f0e9a-7199-437a-b74f-ea3c7a46be40 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:29.605327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f0e9a-7199-437a-b74f-ea3c7a46be40 None admin] PATH_INFO: `/v3/users/835f16cacf6b44ca9b2bcc6e5b0f91e1` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:29.606409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f0e9a-7199-437a-b74f-ea3c7a46be40 None admin] RBAC: Authorizing `identity:get_user(user_id=835f16cacf6b44ca9b2bcc6e5b0f91e1)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:29.607837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f0e9a-7199-437a-b74f-ea3c7a46be40 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:29.609363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 576/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:29 2022] GET /identity/v3/users/835f16cacf6b44ca9b2bcc6e5b0f91e1 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:29.615089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782a91c-9002-474e-8f53-6bdb860cc52e None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:29.708844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782a91c-9002-474e-8f53-6bdb860cc52e None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:29.724949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:29.740940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c1c78-7909-40df-bc00-1181f32d5463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:29.741540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c1c78-7909-40df-bc00-1181f32d5463 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:29.741813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c1c78-7909-40df-bc00-1181f32d5463 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:29.742078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c1c78-7909-40df-bc00-1181f32d5463 None admin] PATH_INFO: `/v3/projects/47304b67d84b46e8a0278d30cffe02ff` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:29.743180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c1c78-7909-40df-bc00-1181f32d5463 None admin] RBAC: Authorizing `identity:get_project(project_id=47304b67d84b46e8a0278d30cffe02ff)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:29.744843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c1c78-7909-40df-bc00-1181f32d5463 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:29.746406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 576/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:29 2022] GET /identity/v3/projects/47304b67d84b46e8a0278d30cffe02ff => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:29.752241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f0e9a-7199-437a-b74f-ea3c7a46be40 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:29.846732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f0e9a-7199-437a-b74f-ea3c7a46be40 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:29.877990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec79fba-2993-469d-83c2-8b53ccba31fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:29.878615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec79fba-2993-469d-83c2-8b53ccba31fa None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:29.878889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec79fba-2993-469d-83c2-8b53ccba31fa None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:29.879182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec79fba-2993-469d-83c2-8b53ccba31fa None admin] PATH_INFO: `/v3/projects/47304b67d84b46e8a0278d30cffe02ff/users/835f16cacf6b44ca9b2bcc6e5b0f91e1/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:29.881525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec79fba-2993-469d-83c2-8b53ccba31fa None admin] RBAC: Authorizing `identity:create_grant(project_id=47304b67d84b46e8a0278d30cffe02ff, user_id=835f16cacf6b44ca9b2bcc6e5b0f91e1, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:29.882931 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec79fba-2993-469d-83c2-8b53ccba31fa None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:29.895332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 577/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:52:29 2022] PUT /identity/v3/projects/47304b67d84b46e8a0278d30cffe02ff/users/835f16cacf6b44ca9b2bcc6e5b0f91e1/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:52:31.843777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691b04-2ab7-4ad2-a1b2-f8199cd40ac5 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:31.844284 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691b04-2ab7-4ad2-a1b2-f8199cd40ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:31.844397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691b04-2ab7-4ad2-a1b2-f8199cd40ac5 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:31.845637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 577/1154] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:31.850747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5507b6-6c57-4b13-b959-08edeb220fb0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:31.851188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5507b6-6c57-4b13-b959-08edeb220fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:31.851304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5507b6-6c57-4b13-b959-08edeb220fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:31.870078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8e5507b6-6c57-4b13-b959-08edeb220fb0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:31.871255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8e5507b6-6c57-4b13-b959-08edeb220fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:31.915362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:31.935205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 578/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:31.958357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52691b04-2ab7-4ad2-a1b2-f8199cd40ac5 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:32.056596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52691b04-2ab7-4ad2-a1b2-f8199cd40ac5 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:32.089105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c3420-6a39-4545-ab49-72b9538526ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:32.089726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c3420-6a39-4545-ab49-72b9538526ae None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:32.090003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c3420-6a39-4545-ab49-72b9538526ae None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:32.090271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c3420-6a39-4545-ab49-72b9538526ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:32.101855 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c3420-6a39-4545-ab49-72b9538526ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-835c3420-6a39-4545-ab49-72b9538526ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:32.106338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:32.107448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:32.108715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 578/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:52:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:32.113449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5507b6-6c57-4b13-b959-08edeb220fb0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:32.206925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5507b6-6c57-4b13-b959-08edeb220fb0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:32.238700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e6005-a199-44a3-89b3-b5e6b66ae614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:32.239304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e6005-a199-44a3-89b3-b5e6b66ae614 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:32.239579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e6005-a199-44a3-89b3-b5e6b66ae614 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:32.239846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e6005-a199-44a3-89b3-b5e6b66ae614 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:32.241043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e6005-a199-44a3-89b3-b5e6b66ae614 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:32.241338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e6005-a199-44a3-89b3-b5e6b66ae614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:32.242672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e6005-a199-44a3-89b3-b5e6b66ae614 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:32.250470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 579/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:52:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:32.256143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c3420-6a39-4545-ab49-72b9538526ae None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:32.300318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:32.352350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c3420-6a39-4545-ab49-72b9538526ae None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:32.383946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12a5b-cfe6-49fd-ab8f-2b1f694d146d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:32.384549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12a5b-cfe6-49fd-ab8f-2b1f694d146d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:32.384824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12a5b-cfe6-49fd-ab8f-2b1f694d146d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:32.385090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12a5b-cfe6-49fd-ab8f-2b1f694d146d None admin] PATH_INFO: `/v3/users/835f16cacf6b44ca9b2bcc6e5b0f91e1` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:32.386174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12a5b-cfe6-49fd-ab8f-2b1f694d146d None admin] RBAC: Authorizing `identity:get_user(user_id=835f16cacf6b44ca9b2bcc6e5b0f91e1)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:32.387616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12a5b-cfe6-49fd-ab8f-2b1f694d146d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:32.389148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 579/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:32 2022] GET /identity/v3/users/835f16cacf6b44ca9b2bcc6e5b0f91e1 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:32.394723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e6005-a199-44a3-89b3-b5e6b66ae614 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:32.439085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:32.491257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e6005-a199-44a3-89b3-b5e6b66ae614 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:32.523653 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a85f35-0e26-485c-ac5c-d9c7a8471837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:32.524252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a85f35-0e26-485c-ac5c-d9c7a8471837 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:32.524533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a85f35-0e26-485c-ac5c-d9c7a8471837 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:32.524806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a85f35-0e26-485c-ac5c-d9c7a8471837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:32.588469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a85f35-0e26-485c-ac5c-d9c7a8471837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:32.589893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a85f35-0e26-485c-ac5c-d9c7a8471837 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:32.631682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:32.684316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 580/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:52:32.690264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12a5b-cfe6-49fd-ab8f-2b1f694d146d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:32.786417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12a5b-cfe6-49fd-ab8f-2b1f694d146d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:32.799291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:32.819541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e4e86-476a-4dd3-bdb1-3b5cffe4ea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:32.820159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4e86-476a-4dd3-bdb1-3b5cffe4ea4e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:32.820439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4e86-476a-4dd3-bdb1-3b5cffe4ea4e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:32.820712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4e86-476a-4dd3-bdb1-3b5cffe4ea4e None admin] PATH_INFO: `/v3/projects/47304b67d84b46e8a0278d30cffe02ff` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:32.821804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e4e86-476a-4dd3-bdb1-3b5cffe4ea4e None admin] RBAC: Authorizing `identity:get_project(project_id=47304b67d84b46e8a0278d30cffe02ff)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:32.823266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e4e86-476a-4dd3-bdb1-3b5cffe4ea4e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:32.824846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 580/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:32 2022] GET /identity/v3/projects/47304b67d84b46e8a0278d30cffe02ff => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:32.830694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a85f35-0e26-485c-ac5c-d9c7a8471837 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:32.925041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a85f35-0e26-485c-ac5c-d9c7a8471837 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:32.957364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87bb56-a460-42d1-bc4e-5126ee54f017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:32.957946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87bb56-a460-42d1-bc4e-5126ee54f017 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:32.958219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87bb56-a460-42d1-bc4e-5126ee54f017 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:32.958484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87bb56-a460-42d1-bc4e-5126ee54f017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:32.959818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87bb56-a460-42d1-bc4e-5126ee54f017 None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=47304b67d84b46e8a0278d30cffe02ff, user.id=835f16cacf6b44ca9b2bcc6e5b0f91e1) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:32.960122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87bb56-a460-42d1-bc4e-5126ee54f017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:32.961518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87bb56-a460-42d1-bc4e-5126ee54f017 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:32.966943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 581/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:32 2022] GET /identity/v3/role_assignments?user.id=835f16cacf6b44ca9b2bcc6e5b0f91e1&scope.project.id=47304b67d84b46e8a0278d30cffe02ff&role.id=8417343e9ef64991bcf51bb9692da102 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:34.965892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9aac6-e343-4acf-bc3c-73808a31dc7a None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:34.966413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9aac6-e343-4acf-bc3c-73808a31dc7a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:34.966413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9aac6-e343-4acf-bc3c-73808a31dc7a None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:34.967921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 581/1162] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:34.973059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d360c-bb7e-40e1-81ba-72de2b314a8d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:34.973475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d360c-bb7e-40e1-81ba-72de2b314a8d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:34.973607 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d360c-bb7e-40e1-81ba-72de2b314a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:34.992114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-556d360c-bb7e-40e1-81ba-72de2b314a8d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:34.993278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-556d360c-bb7e-40e1-81ba-72de2b314a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:34.995486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:35.058916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 582/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:35.080472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9aac6-e343-4acf-bc3c-73808a31dc7a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:35.189112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9aac6-e343-4acf-bc3c-73808a31dc7a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:35.220033 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e3dd6-c40c-423b-9552-83fb39be6b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:35.220553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e3dd6-c40c-423b-9552-83fb39be6b50 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:35.220826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e3dd6-c40c-423b-9552-83fb39be6b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:35.221093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e3dd6-c40c-423b-9552-83fb39be6b50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:35.267340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:35.285018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e3dd6-c40c-423b-9552-83fb39be6b50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:35.286439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e3dd6-c40c-423b-9552-83fb39be6b50 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:35.379617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 582/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:52:35.385521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d360c-bb7e-40e1-81ba-72de2b314a8d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:35.481772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d360c-bb7e-40e1-81ba-72de2b314a8d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:35.513637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbe03a-5377-4fac-9ca3-ac058e2dea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:35.514264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbe03a-5377-4fac-9ca3-ac058e2dea44 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:35.514546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbe03a-5377-4fac-9ca3-ac058e2dea44 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:35.514818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbe03a-5377-4fac-9ca3-ac058e2dea44 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:35.521842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbe03a-5377-4fac-9ca3-ac058e2dea44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:35.523288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbe03a-5377-4fac-9ca3-ac058e2dea44 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:35.524939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-5abbe03a-5377-4fac-9ca3-ac058e2dea44 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:35.535292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:35.536491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:35.537899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 583/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:52:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:35.543075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e3dd6-c40c-423b-9552-83fb39be6b50 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:35.619469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:35.636818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e3dd6-c40c-423b-9552-83fb39be6b50 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:35.669884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f943ad2-6c42-4182-9326-953414c98822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:35.670469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f943ad2-6c42-4182-9326-953414c98822 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:35.670742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f943ad2-6c42-4182-9326-953414c98822 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:35.671035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f943ad2-6c42-4182-9326-953414c98822 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:35.671659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f943ad2-6c42-4182-9326-953414c98822 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:35.671954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f943ad2-6c42-4182-9326-953414c98822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:35.673295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f943ad2-6c42-4182-9326-953414c98822 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:35.680086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 583/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 10 09:52:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:37.658289 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde44aa4-021c-4922-bfca-dc93d0860146 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:37.658895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde44aa4-021c-4922-bfca-dc93d0860146 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:37.658895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde44aa4-021c-4922-bfca-dc93d0860146 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:37.660242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 584/1167] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:37.665330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0136f-122b-4d67-a1de-51d38ddd51c0 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:37.665756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0136f-122b-4d67-a1de-51d38ddd51c0 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:37.665881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0136f-122b-4d67-a1de-51d38ddd51c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:37.684439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-06c0136f-122b-4d67-a1de-51d38ddd51c0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:37.685597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-06c0136f-122b-4d67-a1de-51d38ddd51c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:37.748519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 584/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:37.771154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde44aa4-021c-4922-bfca-dc93d0860146 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:37.866751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde44aa4-021c-4922-bfca-dc93d0860146 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:37.899333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883746b-3a1b-4d1e-8b94-8afec7e15f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:37.899911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883746b-3a1b-4d1e-8b94-8afec7e15f1a None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:37.900184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883746b-3a1b-4d1e-8b94-8afec7e15f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:37.900458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883746b-3a1b-4d1e-8b94-8afec7e15f1a None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:37.901101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883746b-3a1b-4d1e-8b94-8afec7e15f1a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:37.902435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883746b-3a1b-4d1e-8b94-8afec7e15f1a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:37.917691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 585/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Wed Aug 10 09:52:37 2022] POST /identity/v3/domains => generated 267 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:52:39.919353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49ac19-9b6f-40c5-a49f-5331c34bf4db None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:39.919900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49ac19-9b6f-40c5-a49f-5331c34bf4db None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:39.919900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49ac19-9b6f-40c5-a49f-5331c34bf4db None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:39.921189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 585/1170] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:39.926207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e3445-f107-4ff6-87f6-bbaaa728ca2d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:39.926618 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e3445-f107-4ff6-87f6-bbaaa728ca2d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:39.926753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e3445-f107-4ff6-87f6-bbaaa728ca2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:39.944070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c17e3445-f107-4ff6-87f6-bbaaa728ca2d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:39.945242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c17e3445-f107-4ff6-87f6-bbaaa728ca2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:39.996125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:40.009734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 586/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:40.032751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49ac19-9b6f-40c5-a49f-5331c34bf4db None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:40.106204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:40.129495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49ac19-9b6f-40c5-a49f-5331c34bf4db None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:40.161558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dca2e-586f-4093-bf6f-45579bd510b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:40.162739 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dca2e-586f-4093-bf6f-45579bd510b2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:40.163035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dca2e-586f-4093-bf6f-45579bd510b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:40.163299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dca2e-586f-4093-bf6f-45579bd510b2 None admin] PATH_INFO: `/v3/domains/439cf1efdfbd4446a8e0ce9a1bc23424` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:40.173161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dca2e-586f-4093-bf6f-45579bd510b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=439cf1efdfbd4446a8e0ce9a1bc23424)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:40.174528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dca2e-586f-4093-bf6f-45579bd510b2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:40.176091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 586/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Aug 10 09:52:40 2022] GET /identity/v3/domains/439cf1efdfbd4446a8e0ce9a1bc23424 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:40.181752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e3445-f107-4ff6-87f6-bbaaa728ca2d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:40.276275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e3445-f107-4ff6-87f6-bbaaa728ca2d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:40.307765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae8c58-e87f-4c27-8d2b-ad347fb5e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:40.308352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8c58-e87f-4c27-8d2b-ad347fb5e4a4 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:40.308625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8c58-e87f-4c27-8d2b-ad347fb5e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:40.308890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8c58-e87f-4c27-8d2b-ad347fb5e4a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:40.309514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae8c58-e87f-4c27-8d2b-ad347fb5e4a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:40.310901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae8c58-e87f-4c27-8d2b-ad347fb5e4a4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:40.336601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 587/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:52:40 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:52:42.341992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5947c56-a626-4e60-859b-656debea3fbc None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:42.342515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5947c56-a626-4e60-859b-656debea3fbc None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:42.342515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5947c56-a626-4e60-859b-656debea3fbc None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:42.343869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 587/1174] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:42.349225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b42e7-eafa-4bd4-8c73-ad31be674e1e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:42.349589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b42e7-eafa-4bd4-8c73-ad31be674e1e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:42.349774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b42e7-eafa-4bd4-8c73-ad31be674e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:42.367803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3e8b42e7-eafa-4bd4-8c73-ad31be674e1e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:42.369238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3e8b42e7-eafa-4bd4-8c73-ad31be674e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:42.402307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:42.435924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 588/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:42.459884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5947c56-a626-4e60-859b-656debea3fbc None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:42.556813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5947c56-a626-4e60-859b-656debea3fbc None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:42.588094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6f4d6-4fc6-42d2-be4b-46ca1768e937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:42.588705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6f4d6-4fc6-42d2-be4b-46ca1768e937 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:42.588984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6f4d6-4fc6-42d2-be4b-46ca1768e937 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:42.589263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6f4d6-4fc6-42d2-be4b-46ca1768e937 None admin] PATH_INFO: `/v3/domains/439cf1efdfbd4446a8e0ce9a1bc23424` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:42.590363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6f4d6-4fc6-42d2-be4b-46ca1768e937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=439cf1efdfbd4446a8e0ce9a1bc23424)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:42.591815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6f4d6-4fc6-42d2-be4b-46ca1768e937 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:42.593396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 588/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Aug 10 09:52:42 2022] GET /identity/v3/domains/439cf1efdfbd4446a8e0ce9a1bc23424 => generated 267 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:42.598987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b42e7-eafa-4bd4-8c73-ad31be674e1e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:42.693579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b42e7-eafa-4bd4-8c73-ad31be674e1e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:42.726566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3028d-fc5f-42de-81c6-d684bf1d9c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:42.727180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3028d-fc5f-42de-81c6-d684bf1d9c49 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:42.727459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3028d-fc5f-42de-81c6-d684bf1d9c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:42.727728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3028d-fc5f-42de-81c6-d684bf1d9c49 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:42.728369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3028d-fc5f-42de-81c6-d684bf1d9c49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:42.729788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3028d-fc5f-42de-81c6-d684bf1d9c49 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:42.751154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 589/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:52:42 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:52:44.759434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4797b3f-b917-4faa-933f-178407d793a5 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:44.759951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4797b3f-b917-4faa-933f-178407d793a5 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:44.759951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4797b3f-b917-4faa-933f-178407d793a5 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:44.761200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 589/1178] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:44.766560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21ab74-8be4-443b-881c-2ba167f153d1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:44.766987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21ab74-8be4-443b-881c-2ba167f153d1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:44.767142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21ab74-8be4-443b-881c-2ba167f153d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:44.784555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9a21ab74-8be4-443b-881c-2ba167f153d1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:44.785719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9a21ab74-8be4-443b-881c-2ba167f153d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:44.805970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:44.850880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 590/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:44.874698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4797b3f-b917-4faa-933f-178407d793a5 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:44.878415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:44.970570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4797b3f-b917-4faa-933f-178407d793a5 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:45.002890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04ea0e-8af5-447e-a41f-1065f8fcf4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:45.003526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04ea0e-8af5-447e-a41f-1065f8fcf4aa None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:45.003799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04ea0e-8af5-447e-a41f-1065f8fcf4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:45.004066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04ea0e-8af5-447e-a41f-1065f8fcf4aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:45.014100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04ea0e-8af5-447e-a41f-1065f8fcf4aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-bc04ea0e-8af5-447e-a41f-1065f8fcf4aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:45.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:45.019784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:45.020714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 590/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:52:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:45.025527 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21ab74-8be4-443b-881c-2ba167f153d1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:45.120869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21ab74-8be4-443b-881c-2ba167f153d1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:45.157957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d2ea9-dbab-401f-a89b-8368279bb124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:45.158558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d2ea9-dbab-401f-a89b-8368279bb124 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:45.158835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d2ea9-dbab-401f-a89b-8368279bb124 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:45.159140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d2ea9-dbab-401f-a89b-8368279bb124 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:45.159790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d2ea9-dbab-401f-a89b-8368279bb124 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:45.160088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d2ea9-dbab-401f-a89b-8368279bb124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:45.161430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d2ea9-dbab-401f-a89b-8368279bb124 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:45.169739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 591/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:52:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:45.175560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04ea0e-8af5-447e-a41f-1065f8fcf4aa None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:45.234512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:45.271443 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04ea0e-8af5-447e-a41f-1065f8fcf4aa None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:45.303485 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abd64a-30ef-4bb0-8885-2ac07aa90b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:45.304092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abd64a-30ef-4bb0-8885-2ac07aa90b08 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:45.304368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abd64a-30ef-4bb0-8885-2ac07aa90b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:45.304638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abd64a-30ef-4bb0-8885-2ac07aa90b08 None admin] PATH_INFO: `/v3/users/7ec0c3397bf5453a955a4b3e06150706` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:45.317013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abd64a-30ef-4bb0-8885-2ac07aa90b08 None admin] RBAC: Authorizing `identity:get_user(user_id=7ec0c3397bf5453a955a4b3e06150706)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:45.318436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abd64a-30ef-4bb0-8885-2ac07aa90b08 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:45.319997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 591/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:45 2022] GET /identity/v3/users/7ec0c3397bf5453a955a4b3e06150706 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:45.325509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d2ea9-dbab-401f-a89b-8368279bb124 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:45.344484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:45.423929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d2ea9-dbab-401f-a89b-8368279bb124 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:45.456016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264f8e4-101e-457c-a87a-fc55eff84ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:45.456611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264f8e4-101e-457c-a87a-fc55eff84ddc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:45.456890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264f8e4-101e-457c-a87a-fc55eff84ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:45.457160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264f8e4-101e-457c-a87a-fc55eff84ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:45.523538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264f8e4-101e-457c-a87a-fc55eff84ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:45.524895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264f8e4-101e-457c-a87a-fc55eff84ddc None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:45.541548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:45.618857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 592/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:52:45.625028 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abd64a-30ef-4bb0-8885-2ac07aa90b08 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:45.723358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abd64a-30ef-4bb0-8885-2ac07aa90b08 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:45.745407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:45.757936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d060a-0a31-41d8-af2f-596f473e4380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:45.758658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d060a-0a31-41d8-af2f-596f473e4380 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:45.759031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d060a-0a31-41d8-af2f-596f473e4380 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:45.759368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d060a-0a31-41d8-af2f-596f473e4380 None admin] PATH_INFO: `/v3/projects/38d692d32d52403dbfdbb60f25a136f9` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:45.760564 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d060a-0a31-41d8-af2f-596f473e4380 None admin] RBAC: Authorizing `identity:get_project(project_id=38d692d32d52403dbfdbb60f25a136f9)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:45.762043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d060a-0a31-41d8-af2f-596f473e4380 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:45.767308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 592/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:45 2022] GET /identity/v3/projects/38d692d32d52403dbfdbb60f25a136f9 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:45.769929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264f8e4-101e-457c-a87a-fc55eff84ddc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:45.867677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264f8e4-101e-457c-a87a-fc55eff84ddc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:45.884994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:45.901726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1494a-2e07-41ed-bfff-190ed0bf528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:45.902326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1494a-2e07-41ed-bfff-190ed0bf528c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:45.902595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1494a-2e07-41ed-bfff-190ed0bf528c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:45.902866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1494a-2e07-41ed-bfff-190ed0bf528c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:45.903602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1494a-2e07-41ed-bfff-190ed0bf528c None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=38d692d32d52403dbfdbb60f25a136f9, user.id=7ec0c3397bf5453a955a4b3e06150706) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:45.903903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1494a-2e07-41ed-bfff-190ed0bf528c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:45.905308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1494a-2e07-41ed-bfff-190ed0bf528c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:45.910585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 593/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:45 2022] GET /identity/v3/role_assignments?user.id=7ec0c3397bf5453a955a4b3e06150706&scope.project.id=38d692d32d52403dbfdbb60f25a136f9&role.id=8417343e9ef64991bcf51bb9692da102 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:47.845986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae9fe3-831f-4e65-b73a-78ab239955cb None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:47.846523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae9fe3-831f-4e65-b73a-78ab239955cb None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:47.846523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae9fe3-831f-4e65-b73a-78ab239955cb None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:47.847853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 593/1186] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:47.853585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf52cb-316d-4226-ae30-bcf63995eeaa None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:47.854015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf52cb-316d-4226-ae30-bcf63995eeaa None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:47.854134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf52cb-316d-4226-ae30-bcf63995eeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:47.872459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-15cf52cb-316d-4226-ae30-bcf63995eeaa None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:47.873621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-15cf52cb-316d-4226-ae30-bcf63995eeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:47.936598 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 594/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:47.958223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae9fe3-831f-4e65-b73a-78ab239955cb None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:48.054236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae9fe3-831f-4e65-b73a-78ab239955cb None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:48.085913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e376636-09c8-4fe7-acc5-152f5ee04731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:48.086505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e376636-09c8-4fe7-acc5-152f5ee04731 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:48.086776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e376636-09c8-4fe7-acc5-152f5ee04731 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:48.087178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e376636-09c8-4fe7-acc5-152f5ee04731 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:48.096710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e376636-09c8-4fe7-acc5-152f5ee04731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-4e376636-09c8-4fe7-acc5-152f5ee04731 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:48.101173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:48.102300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:52:48.103387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 594/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:52:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:48.108004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf52cb-316d-4226-ae30-bcf63995eeaa None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:48.201923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf52cb-316d-4226-ae30-bcf63995eeaa None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:48.214255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:48.234291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655571c-8012-4c5a-9aa8-5696cf39df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:48.234880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655571c-8012-4c5a-9aa8-5696cf39df29 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:48.235185 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655571c-8012-4c5a-9aa8-5696cf39df29 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:48.235455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655571c-8012-4c5a-9aa8-5696cf39df29 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:48.236104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655571c-8012-4c5a-9aa8-5696cf39df29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:48.236402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655571c-8012-4c5a-9aa8-5696cf39df29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:48.237748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655571c-8012-4c5a-9aa8-5696cf39df29 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:48.246161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 595/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:52:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:52:48.252064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e376636-09c8-4fe7-acc5-152f5ee04731 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:48.303414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:48.351281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e376636-09c8-4fe7-acc5-152f5ee04731 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:48.384405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66795d6-8780-4d6a-89a7-e5742a445894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:48.385011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66795d6-8780-4d6a-89a7-e5742a445894 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:48.385283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66795d6-8780-4d6a-89a7-e5742a445894 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:48.385550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66795d6-8780-4d6a-89a7-e5742a445894 None admin] PATH_INFO: `/v3/users/7ec0c3397bf5453a955a4b3e06150706` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:48.386635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66795d6-8780-4d6a-89a7-e5742a445894 None admin] RBAC: Authorizing `identity:get_user(user_id=7ec0c3397bf5453a955a4b3e06150706)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:48.388074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66795d6-8780-4d6a-89a7-e5742a445894 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:48.389605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 595/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:48 2022] GET /identity/v3/users/7ec0c3397bf5453a955a4b3e06150706 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:48.395346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655571c-8012-4c5a-9aa8-5696cf39df29 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:48.490838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655571c-8012-4c5a-9aa8-5696cf39df29 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:48.522142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5122db2-e448-424e-ae71-79ad9cf16897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:48.522756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5122db2-e448-424e-ae71-79ad9cf16897 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:48.523063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5122db2-e448-424e-ae71-79ad9cf16897 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:48.523338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5122db2-e448-424e-ae71-79ad9cf16897 None admin] PATH_INFO: `/v3/projects/38d692d32d52403dbfdbb60f25a136f9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:48.524436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5122db2-e448-424e-ae71-79ad9cf16897 None admin] RBAC: Authorizing `identity:get_project(project_id=38d692d32d52403dbfdbb60f25a136f9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:48.525843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5122db2-e448-424e-ae71-79ad9cf16897 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:48.527485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 596/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:48 2022] GET /identity/v3/projects/38d692d32d52403dbfdbb60f25a136f9 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:48.533311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66795d6-8780-4d6a-89a7-e5742a445894 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:48.627135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66795d6-8780-4d6a-89a7-e5742a445894 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:48.643322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:48.659702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e1d83-d6e9-4da0-bbb0-82c756ecf55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:48.660339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e1d83-d6e9-4da0-bbb0-82c756ecf55d None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:48.660614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e1d83-d6e9-4da0-bbb0-82c756ecf55d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:48.660883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e1d83-d6e9-4da0-bbb0-82c756ecf55d None admin] PATH_INFO: `/v3/projects/38d692d32d52403dbfdbb60f25a136f9/users/7ec0c3397bf5453a955a4b3e06150706/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:48.663283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e1d83-d6e9-4da0-bbb0-82c756ecf55d None admin] RBAC: Authorizing `identity:create_grant(project_id=38d692d32d52403dbfdbb60f25a136f9, user_id=7ec0c3397bf5453a955a4b3e06150706, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:48.664708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e1d83-d6e9-4da0-bbb0-82c756ecf55d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:48.676428 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 596/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:52:48 2022] PUT /identity/v3/projects/38d692d32d52403dbfdbb60f25a136f9/users/7ec0c3397bf5453a955a4b3e06150706/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:52:50.660716 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2a82b-2d05-41d4-a04c-8b0929b02e3f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:50.661226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2a82b-2d05-41d4-a04c-8b0929b02e3f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:50.661226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2a82b-2d05-41d4-a04c-8b0929b02e3f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:50.663115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 597/1193] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:52:50.668210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a7f28-a70d-41ed-90af-9710d29532b3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:50.668574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a7f28-a70d-41ed-90af-9710d29532b3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:50.668760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a7f28-a70d-41ed-90af-9710d29532b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:50.686242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-8e7a7f28-a70d-41ed-90af-9710d29532b3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:52:50.687460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8e7a7f28-a70d-41ed-90af-9710d29532b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:52:50.750999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 597/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:52:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:52:50.774422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2a82b-2d05-41d4-a04c-8b0929b02e3f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:50.816202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:50.872369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2a82b-2d05-41d4-a04c-8b0929b02e3f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:50.908113 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de5c91-a1f2-4f8a-b6d1-97ef67e516cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:50.909065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de5c91-a1f2-4f8a-b6d1-97ef67e516cc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:50.909510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de5c91-a1f2-4f8a-b6d1-97ef67e516cc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:50.909930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de5c91-a1f2-4f8a-b6d1-97ef67e516cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:50.923934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de5c91-a1f2-4f8a-b6d1-97ef67e516cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-63de5c91-a1f2-4f8a-b6d1-97ef67e516cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:52:50.931482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:52:50.932546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:52:50.933552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 598/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:52:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:52:50.941318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a7f28-a70d-41ed-90af-9710d29532b3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:51.034975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a7f28-a70d-41ed-90af-9710d29532b3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:51.044219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:51.069947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662b619-1e23-49f1-aea3-b840eb65cc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:51.070545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662b619-1e23-49f1-aea3-b840eb65cc89 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:51.070823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662b619-1e23-49f1-aea3-b840eb65cc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:51.071124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662b619-1e23-49f1-aea3-b840eb65cc89 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:51.071778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662b619-1e23-49f1-aea3-b840eb65cc89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:51.072079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662b619-1e23-49f1-aea3-b840eb65cc89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:51.073450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662b619-1e23-49f1-aea3-b840eb65cc89 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:51.081110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 598/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:52:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:51.087111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de5c91-a1f2-4f8a-b6d1-97ef67e516cc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:51.184367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de5c91-a1f2-4f8a-b6d1-97ef67e516cc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:51.185769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:51.217579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6e2c9-6fe4-4aa0-bdbc-7bbb8cc84810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:51.218184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6e2c9-6fe4-4aa0-bdbc-7bbb8cc84810 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:51.218459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6e2c9-6fe4-4aa0-bdbc-7bbb8cc84810 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:51.218722 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6e2c9-6fe4-4aa0-bdbc-7bbb8cc84810 None admin] PATH_INFO: `/v3/users/7ec0c3397bf5453a955a4b3e06150706` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:51.219833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6e2c9-6fe4-4aa0-bdbc-7bbb8cc84810 None admin] RBAC: Authorizing `identity:get_user(user_id=7ec0c3397bf5453a955a4b3e06150706)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:51.221221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6e2c9-6fe4-4aa0-bdbc-7bbb8cc84810 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:51.222744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 599/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 10 09:52:51 2022] GET /identity/v3/users/7ec0c3397bf5453a955a4b3e06150706 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:51.228615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662b619-1e23-49f1-aea3-b840eb65cc89 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:51.324513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662b619-1e23-49f1-aea3-b840eb65cc89 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:51.355375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c818a-b25d-4497-a4be-c9552a11bfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:51.355966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c818a-b25d-4497-a4be-c9552a11bfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:51.356236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c818a-b25d-4497-a4be-c9552a11bfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:51.356501 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c818a-b25d-4497-a4be-c9552a11bfa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:51.393374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:51.420950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c818a-b25d-4497-a4be-c9552a11bfa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:51.422381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c818a-b25d-4497-a4be-c9552a11bfa5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:51.515967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 599/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:52:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:52:51.522320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6e2c9-6fe4-4aa0-bdbc-7bbb8cc84810 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:51.619938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6e2c9-6fe4-4aa0-bdbc-7bbb8cc84810 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:51.653878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79479da-c657-4316-8c90-af9a72603399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:51.654499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79479da-c657-4316-8c90-af9a72603399 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:51.654771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79479da-c657-4316-8c90-af9a72603399 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:51.655067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79479da-c657-4316-8c90-af9a72603399 None admin] PATH_INFO: `/v3/projects/38d692d32d52403dbfdbb60f25a136f9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:51.656159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79479da-c657-4316-8c90-af9a72603399 None admin] RBAC: Authorizing `identity:get_project(project_id=38d692d32d52403dbfdbb60f25a136f9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:51.657562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79479da-c657-4316-8c90-af9a72603399 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:51.659201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 600/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 10 09:52:51 2022] GET /identity/v3/projects/38d692d32d52403dbfdbb60f25a136f9 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:52:51.665938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c818a-b25d-4497-a4be-c9552a11bfa5 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:52:51.731917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:52:51.762516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c818a-b25d-4497-a4be-c9552a11bfa5 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:52:51.794959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c423c-7d06-49a1-ba21-f48067424ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:52:51.795567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c423c-7d06-49a1-ba21-f48067424ced None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:52:51.795836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c423c-7d06-49a1-ba21-f48067424ced None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:52:51.796104 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c423c-7d06-49a1-ba21-f48067424ced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:52:51.796808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c423c-7d06-49a1-ba21-f48067424ced None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=38d692d32d52403dbfdbb60f25a136f9, user.id=7ec0c3397bf5453a955a4b3e06150706) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:52:51.797107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c423c-7d06-49a1-ba21-f48067424ced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:52:51.798505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c423c-7d06-49a1-ba21-f48067424ced None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:52:51.804892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 600/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:52:51 2022] GET /identity/v3/role_assignments?user.id=7ec0c3397bf5453a955a4b3e06150706&scope.project.id=38d692d32d52403dbfdbb60f25a136f9&role.id=8417343e9ef64991bcf51bb9692da102 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:01.649085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=81 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:01.801455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=33 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:46.461844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7403e8-fcc3-4a73-bd65-2609f10c370e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:46.461844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7403e8-fcc3-4a73-bd65-2609f10c370e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:46.461844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7403e8-fcc3-4a73-bd65-2609f10c370e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:46.461844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 601/1201] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:53:46.468612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a3bf0-e9ae-4b5a-ad29-2b4347a792f8 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:46.469421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a3bf0-e9ae-4b5a-ad29-2b4347a792f8 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:46.469849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a3bf0-e9ae-4b5a-ad29-2b4347a792f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:46.498153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ab1a3bf0-e9ae-4b5a-ad29-2b4347a792f8 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:53:46.499390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ab1a3bf0-e9ae-4b5a-ad29-2b4347a792f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:53:46.587461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 601/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:53:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:53:46.618559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7403e8-fcc3-4a73-bd65-2609f10c370e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:46.759233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7403e8-fcc3-4a73-bd65-2609f10c370e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:46.802556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538da3e1-516f-4b0f-bc13-602b9f31fd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:46.803207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538da3e1-516f-4b0f-bc13-602b9f31fd5c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:46.807250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538da3e1-516f-4b0f-bc13-602b9f31fd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:46.807250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538da3e1-516f-4b0f-bc13-602b9f31fd5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:46.818295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538da3e1-516f-4b0f-bc13-602b9f31fd5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:46.819967 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538da3e1-516f-4b0f-bc13-602b9f31fd5c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-538da3e1-516f-4b0f-bc13-602b9f31fd5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:53:46.839659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:46.846539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:53:46.848104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:53:46.848104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:46.848104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:53:46.848104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 602/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:46.848347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a3bf0-e9ae-4b5a-ad29-2b4347a792f8 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:46.982794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a3bf0-e9ae-4b5a-ad29-2b4347a792f8 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:47.027935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:47.033687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf24c-24e0-458c-b2b2-487b3bafc88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:47.034727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf24c-24e0-458c-b2b2-487b3bafc88f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:47.035147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf24c-24e0-458c-b2b2-487b3bafc88f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:47.035448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf24c-24e0-458c-b2b2-487b3bafc88f None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:47.036115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf24c-24e0-458c-b2b2-487b3bafc88f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:47.036655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf24c-24e0-458c-b2b2-487b3bafc88f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:47.038055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf24c-24e0-458c-b2b2-487b3bafc88f None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:47.056980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 602/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:47.063223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538da3e1-516f-4b0f-bc13-602b9f31fd5c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:47.134506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:47.193083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538da3e1-516f-4b0f-bc13-602b9f31fd5c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:47.234645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34d1b0-4a04-4979-8367-bfab0d87f828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:47.235357 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34d1b0-4a04-4979-8367-bfab0d87f828 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:47.235645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34d1b0-4a04-4979-8367-bfab0d87f828 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:47.235922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34d1b0-4a04-4979-8367-bfab0d87f828 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:47.236664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34d1b0-4a04-4979-8367-bfab0d87f828 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:47.238297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34d1b0-4a04-4979-8367-bfab0d87f828 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:47.275331 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 603/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 10 09:53:47 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:53:49.622969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8e7c6-73a7-41c3-a46b-c98bbdfa50d1 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:49.623508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8e7c6-73a7-41c3-a46b-c98bbdfa50d1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:49.623618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8e7c6-73a7-41c3-a46b-c98bbdfa50d1 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:49.625059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 603/1206] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:53:49.630885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce169-3086-48eb-bc27-c9ee6382aa05 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:49.631367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce169-3086-48eb-bc27-c9ee6382aa05 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:49.631455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce169-3086-48eb-bc27-c9ee6382aa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:49.651270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-22cce169-3086-48eb-bc27-c9ee6382aa05 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:53:49.652491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-22cce169-3086-48eb-bc27-c9ee6382aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:53:49.729929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 604/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:53:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:53:49.757468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8e7c6-73a7-41c3-a46b-c98bbdfa50d1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:49.879259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8e7c6-73a7-41c3-a46b-c98bbdfa50d1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:49.918989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb153e-b7a7-4de3-a9cb-8dc7019c06b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:49.920096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb153e-b7a7-4de3-a9cb-8dc7019c06b1 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:49.920588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb153e-b7a7-4de3-a9cb-8dc7019c06b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:49.921081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb153e-b7a7-4de3-a9cb-8dc7019c06b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:49.935958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb153e-b7a7-4de3-a9cb-8dc7019c06b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-cdbb153e-b7a7-4de3-a9cb-8dc7019c06b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:53:49.942208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:49.943351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:53:49.944198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 604/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:53:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:49.950029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cce169-3086-48eb-bc27-c9ee6382aa05 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:49.995202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:50.066693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cce169-3086-48eb-bc27-c9ee6382aa05 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:50.107546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08861419-030b-4893-8ecf-eb02c243fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:50.108228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08861419-030b-4893-8ecf-eb02c243fff2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:50.108532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08861419-030b-4893-8ecf-eb02c243fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:50.108832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08861419-030b-4893-8ecf-eb02c243fff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:50.109527 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08861419-030b-4893-8ecf-eb02c243fff2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:50.109825 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08861419-030b-4893-8ecf-eb02c243fff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:50.111295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08861419-030b-4893-8ecf-eb02c243fff2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:50.121877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 605/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:53:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:50.129196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb153e-b7a7-4de3-a9cb-8dc7019c06b1 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:50.192419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:50.251277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb153e-b7a7-4de3-a9cb-8dc7019c06b1 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:50.292207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0c26e-b770-4c70-b51c-3b64c658c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:50.292840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0c26e-b770-4c70-b51c-3b64c658c5cd None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:50.293145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0c26e-b770-4c70-b51c-3b64c658c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:50.293395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0c26e-b770-4c70-b51c-3b64c658c5cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:50.304790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0c26e-b770-4c70-b51c-3b64c658c5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:50.306242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0c26e-b770-4c70-b51c-3b64c658c5cd None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-17c0c26e-b770-4c70-b51c-3b64c658c5cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:53:50.323285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:50.324782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:53:50.326516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:53:50.326516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:50.326516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:53:50.326516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 605/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:50.331136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08861419-030b-4893-8ecf-eb02c243fff2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:50.435165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:50.438980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08861419-030b-4893-8ecf-eb02c243fff2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:50.477265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc22f63-6e15-4c9b-8904-80d5b4b60883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:50.477974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc22f63-6e15-4c9b-8904-80d5b4b60883 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:50.478255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc22f63-6e15-4c9b-8904-80d5b4b60883 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:50.478528 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc22f63-6e15-4c9b-8904-80d5b4b60883 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:50.479193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc22f63-6e15-4c9b-8904-80d5b4b60883 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:50.479496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc22f63-6e15-4c9b-8904-80d5b4b60883 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:50.480857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc22f63-6e15-4c9b-8904-80d5b4b60883 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:50.493779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 606/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:50.499783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0c26e-b770-4c70-b51c-3b64c658c5cd None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:50.620676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0c26e-b770-4c70-b51c-3b64c658c5cd None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:50.622193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:50.661151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835ceb6-9e75-4e7c-a278-c337f89af980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:50.661933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835ceb6-9e75-4e7c-a278-c337f89af980 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:50.662044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835ceb6-9e75-4e7c-a278-c337f89af980 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:50.662284 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835ceb6-9e75-4e7c-a278-c337f89af980 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:50.669995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835ceb6-9e75-4e7c-a278-c337f89af980 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:50.671406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835ceb6-9e75-4e7c-a278-c337f89af980 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-c835ceb6-9e75-4e7c-a278-c337f89af980 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:53:50.682740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:50.688751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:53:50.689949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 606/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:53:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:50.691562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc22f63-6e15-4c9b-8904-80d5b4b60883 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:50.808847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc22f63-6e15-4c9b-8904-80d5b4b60883 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:50.850390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6aefa-e76e-4cb3-ad3b-9b080449aa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:50.851287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6aefa-e76e-4cb3-ad3b-9b080449aa93 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:50.851745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6aefa-e76e-4cb3-ad3b-9b080449aa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:50.852183 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6aefa-e76e-4cb3-ad3b-9b080449aa93 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:50.852933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d6aefa-e76e-4cb3-ad3b-9b080449aa93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:50.853222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d6aefa-e76e-4cb3-ad3b-9b080449aa93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:50.854645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d6aefa-e76e-4cb3-ad3b-9b080449aa93 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:50.879043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 607/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 10 09:53:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:53:50.881773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835ceb6-9e75-4e7c-a278-c337f89af980 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:50.995988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835ceb6-9e75-4e7c-a278-c337f89af980 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:51.040422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe17bb4-e557-4aee-b44d-d8c9e2cebecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:51.041057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe17bb4-e557-4aee-b44d-d8c9e2cebecb None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:51.041376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe17bb4-e557-4aee-b44d-d8c9e2cebecb None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:51.041714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe17bb4-e557-4aee-b44d-d8c9e2cebecb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:51.083176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:51.130726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe17bb4-e557-4aee-b44d-d8c9e2cebecb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:51.132250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe17bb4-e557-4aee-b44d-d8c9e2cebecb None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:51.264163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 607/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:53:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:53:51.265707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6aefa-e76e-4cb3-ad3b-9b080449aa93 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:51.287506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:51.402406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6aefa-e76e-4cb3-ad3b-9b080449aa93 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:51.442195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba8eac-fa67-4b3c-b9be-85c68c731e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:51.442827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba8eac-fa67-4b3c-b9be-85c68c731e4f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:51.443133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba8eac-fa67-4b3c-b9be-85c68c731e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:51.443400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba8eac-fa67-4b3c-b9be-85c68c731e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:51.454936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba8eac-fa67-4b3c-b9be-85c68c731e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:51.457450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba8eac-fa67-4b3c-b9be-85c68c731e4f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-73ba8eac-fa67-4b3c-b9be-85c68c731e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:53:51.475604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:51.476580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:53:51.477639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:53:51.477639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:51.477639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:53:51.477639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 608/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:51.482993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe17bb4-e557-4aee-b44d-d8c9e2cebecb None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:51.550370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:51.599116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe17bb4-e557-4aee-b44d-d8c9e2cebecb None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:51.638663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7868d-67bc-466b-8aa9-5fe2e5ce5193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:51.639554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7868d-67bc-466b-8aa9-5fe2e5ce5193 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:51.639828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7868d-67bc-466b-8aa9-5fe2e5ce5193 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:51.640221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7868d-67bc-466b-8aa9-5fe2e5ce5193 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:51.641199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7868d-67bc-466b-8aa9-5fe2e5ce5193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:51.641619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7868d-67bc-466b-8aa9-5fe2e5ce5193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:51.643344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7868d-67bc-466b-8aa9-5fe2e5ce5193 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:51.658497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 608/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:53:51.663850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba8eac-fa67-4b3c-b9be-85c68c731e4f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:51.743423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:51.776954 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba8eac-fa67-4b3c-b9be-85c68c731e4f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:51.813181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff28cb-60fe-4473-b11f-cc711e48a55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:51.813804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff28cb-60fe-4473-b11f-cc711e48a55a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:51.814086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff28cb-60fe-4473-b11f-cc711e48a55a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:51.814378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff28cb-60fe-4473-b11f-cc711e48a55a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:51.821747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff28cb-60fe-4473-b11f-cc711e48a55a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:51.823194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff28cb-60fe-4473-b11f-cc711e48a55a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-54ff28cb-60fe-4473-b11f-cc711e48a55a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:53:51.834256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:51.835652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:51.836558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:53:51.836558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:53:51.836558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:53:51.836558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:53:51.836558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:53:51.836558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:53:51.836558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:53:51.836558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:53:51.840454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 609/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:53:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:51.841374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7868d-67bc-466b-8aa9-5fe2e5ce5193 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:51.949430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7868d-67bc-466b-8aa9-5fe2e5ce5193 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:51.975611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:51.992106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1bd45-1762-4237-9ac7-554c9bdaf413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:51.992860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1bd45-1762-4237-9ac7-554c9bdaf413 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:51.993236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1bd45-1762-4237-9ac7-554c9bdaf413 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:51.993604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1bd45-1762-4237-9ac7-554c9bdaf413 None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:51.994661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1bd45-1762-4237-9ac7-554c9bdaf413 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:51.995084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1bd45-1762-4237-9ac7-554c9bdaf413 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:51.996590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1bd45-1762-4237-9ac7-554c9bdaf413 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:52.010021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 609/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:53:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:52.016106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff28cb-60fe-4473-b11f-cc711e48a55a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:52.134060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff28cb-60fe-4473-b11f-cc711e48a55a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:52.170023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:52.179348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63997b9f-65ec-4b5d-81a0-5af8b798f831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:52.180227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997b9f-65ec-4b5d-81a0-5af8b798f831 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:52.180503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997b9f-65ec-4b5d-81a0-5af8b798f831 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:52.180776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997b9f-65ec-4b5d-81a0-5af8b798f831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:52.181545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63997b9f-65ec-4b5d-81a0-5af8b798f831 None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=d3d7b3cb498a4da18e203456e09b59e7) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:52.181810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63997b9f-65ec-4b5d-81a0-5af8b798f831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:52.187007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63997b9f-65ec-4b5d-81a0-5af8b798f831 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:52.203068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 610/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:53:52 2022] GET /identity/v3/role_assignments?user.id=d3d7b3cb498a4da18e203456e09b59e7&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:53:54.438868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc226-b453-4269-9ec6-f927749642e9 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:54.439427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc226-b453-4269-9ec6-f927749642e9 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:54.439427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc226-b453-4269-9ec6-f927749642e9 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:54.441577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 610/1220] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:53:54.447127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd49a7a-0775-4b35-a135-4e39959945cc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:54.447590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd49a7a-0775-4b35-a135-4e39959945cc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:54.447681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd49a7a-0775-4b35-a135-4e39959945cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:54.466073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5fd49a7a-0775-4b35-a135-4e39959945cc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:53:54.467321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5fd49a7a-0775-4b35-a135-4e39959945cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:53:54.545379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 611/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:53:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:53:54.569181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afc226-b453-4269-9ec6-f927749642e9 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:54.683245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afc226-b453-4269-9ec6-f927749642e9 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:54.719842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bf915-325b-446e-b6fb-3296845fb5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:54.720636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bf915-325b-446e-b6fb-3296845fb5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:54.720943 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bf915-325b-446e-b6fb-3296845fb5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:54.721232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bf915-325b-446e-b6fb-3296845fb5a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:54.732674 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bf915-325b-446e-b6fb-3296845fb5a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-713bf915-325b-446e-b6fb-3296845fb5a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:53:54.737432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:54.738527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:53:54.739533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 611/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:53:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:54.744959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd49a7a-0775-4b35-a135-4e39959945cc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:54.855763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd49a7a-0775-4b35-a135-4e39959945cc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:54.880322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:54.893405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16649113-dca7-4d71-9d36-a882dfbca530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:54.894023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16649113-dca7-4d71-9d36-a882dfbca530 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:54.894307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16649113-dca7-4d71-9d36-a882dfbca530 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:54.894618 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16649113-dca7-4d71-9d36-a882dfbca530 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:54.895512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16649113-dca7-4d71-9d36-a882dfbca530 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:54.895856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16649113-dca7-4d71-9d36-a882dfbca530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:54.897497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16649113-dca7-4d71-9d36-a882dfbca530 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:54.907400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 612/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:53:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:54.913985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bf915-325b-446e-b6fb-3296845fb5a6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:54.959283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:55.017065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bf915-325b-446e-b6fb-3296845fb5a6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:55.054771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7431f-9b02-43ac-a93f-36c49d193f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:55.055576 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7431f-9b02-43ac-a93f-36c49d193f33 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:55.055927 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7431f-9b02-43ac-a93f-36c49d193f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:55.056298 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7431f-9b02-43ac-a93f-36c49d193f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:55.067813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7431f-9b02-43ac-a93f-36c49d193f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:55.069263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7431f-9b02-43ac-a93f-36c49d193f33 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-01e7431f-9b02-43ac-a93f-36c49d193f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:53:55.083926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:55.085031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:53:55.086092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:53:55.086092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:55.086092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:53:55.086301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 612/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:53:55.092086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16649113-dca7-4d71-9d36-a882dfbca530 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:55.211412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16649113-dca7-4d71-9d36-a882dfbca530 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:55.249256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dd78f-4036-4217-9e4b-4e0824901b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:55.249927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dd78f-4036-4217-9e4b-4e0824901b57 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:55.250206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dd78f-4036-4217-9e4b-4e0824901b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:55.250475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dd78f-4036-4217-9e4b-4e0824901b57 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:55.251133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dd78f-4036-4217-9e4b-4e0824901b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:55.251430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dd78f-4036-4217-9e4b-4e0824901b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:55.252779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dd78f-4036-4217-9e4b-4e0824901b57 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:55.265365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 613/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:55.271600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7431f-9b02-43ac-a93f-36c49d193f33 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:55.373089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:55.380746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7431f-9b02-43ac-a93f-36c49d193f33 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:55.419366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3fbc0-ec35-4e13-ba79-22c657a49a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:55.419998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3fbc0-ec35-4e13-ba79-22c657a49a41 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:55.420273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3fbc0-ec35-4e13-ba79-22c657a49a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:55.420545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3fbc0-ec35-4e13-ba79-22c657a49a41 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:55.428810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3fbc0-ec35-4e13-ba79-22c657a49a41 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:55.430679 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3fbc0-ec35-4e13-ba79-22c657a49a41 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-ddf3fbc0-ec35-4e13-ba79-22c657a49a41 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:53:55.444268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:55.445885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:53:55.447073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 613/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:53:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:55.451755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dd78f-4036-4217-9e4b-4e0824901b57 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:55.505255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:55.562193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dd78f-4036-4217-9e4b-4e0824901b57 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:55.601445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b2884-cd7e-457e-bed9-f846661dab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:55.602055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b2884-cd7e-457e-bed9-f846661dab1b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:55.602382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b2884-cd7e-457e-bed9-f846661dab1b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:55.602656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b2884-cd7e-457e-bed9-f846661dab1b None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:55.603631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b2884-cd7e-457e-bed9-f846661dab1b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:55.603939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b2884-cd7e-457e-bed9-f846661dab1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:55.605834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b2884-cd7e-457e-bed9-f846661dab1b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:55.625223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 614/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 10 09:53:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:53:55.630986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3fbc0-ec35-4e13-ba79-22c657a49a41 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:55.734753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3fbc0-ec35-4e13-ba79-22c657a49a41 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:55.770929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3239f-85eb-4386-9dd9-f207ebd56b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:55.771509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3239f-85eb-4386-9dd9-f207ebd56b8d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:55.771796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3239f-85eb-4386-9dd9-f207ebd56b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:55.772046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3239f-85eb-4386-9dd9-f207ebd56b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:55.785275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3239f-85eb-4386-9dd9-f207ebd56b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:55.787165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3239f-85eb-4386-9dd9-f207ebd56b8d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-0bf3239f-85eb-4386-9dd9-f207ebd56b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:53:55.803494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:55.804703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:53:55.805791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:53:55.805791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:55.805791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:53:55.805791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 614/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:55.810642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b2884-cd7e-457e-bed9-f846661dab1b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:55.917758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b2884-cd7e-457e-bed9-f846661dab1b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:55.920086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:55.956186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafd48d-bbbc-4301-b824-acbdfd80187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:55.957508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafd48d-bbbc-4301-b824-acbdfd80187c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:55.957790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafd48d-bbbc-4301-b824-acbdfd80187c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:55.958072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafd48d-bbbc-4301-b824-acbdfd80187c None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:55.958858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafd48d-bbbc-4301-b824-acbdfd80187c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:55.959222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafd48d-bbbc-4301-b824-acbdfd80187c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:55.960990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafd48d-bbbc-4301-b824-acbdfd80187c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:55.978315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 615/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:55.984042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3239f-85eb-4386-9dd9-f207ebd56b8d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:55.993883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:56.095094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3239f-85eb-4386-9dd9-f207ebd56b8d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:56.129696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7e55d-ccd3-46cc-97b3-a38814ca8010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:56.130319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7e55d-ccd3-46cc-97b3-a38814ca8010 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:56.130594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7e55d-ccd3-46cc-97b3-a38814ca8010 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:56.130862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7e55d-ccd3-46cc-97b3-a38814ca8010 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:56.138899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7e55d-ccd3-46cc-97b3-a38814ca8010 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:56.140368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7e55d-ccd3-46cc-97b3-a38814ca8010 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-16e7e55d-ccd3-46cc-97b3-a38814ca8010 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:53:56.155508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:56.156673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:56.157578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:53:56.157578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:53:56.157578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:53:56.157578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:53:56.157578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:53:56.157578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:53:56.157578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:53:56.157578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:53:56.157578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 615/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:53:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:56.163165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafd48d-bbbc-4301-b824-acbdfd80187c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:56.290705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafd48d-bbbc-4301-b824-acbdfd80187c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:56.332745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ce274-f2df-434a-93f4-64627d4e441f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:56.333449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce274-f2df-434a-93f4-64627d4e441f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:56.333812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce274-f2df-434a-93f4-64627d4e441f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:56.334166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce274-f2df-434a-93f4-64627d4e441f None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:56.335081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce274-f2df-434a-93f4-64627d4e441f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:56.335463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce274-f2df-434a-93f4-64627d4e441f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:56.337210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce274-f2df-434a-93f4-64627d4e441f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:56.351851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 616/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:53:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:53:56.358247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7e55d-ccd3-46cc-97b3-a38814ca8010 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:56.436726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:56.475901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7e55d-ccd3-46cc-97b3-a38814ca8010 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:56.515198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1223b-3eb9-4070-bb3b-31a31a9c2788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:56.515844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1223b-3eb9-4070-bb3b-31a31a9c2788 None admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:56.516142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1223b-3eb9-4070-bb3b-31a31a9c2788 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:56.516387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1223b-3eb9-4070-bb3b-31a31a9c2788 None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/d3d7b3cb498a4da18e203456e09b59e7/roles/bacea3b4b26e48a598b75ea1b301fafb` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:56.532421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a1223b-3eb9-4070-bb3b-31a31a9c2788 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=d3d7b3cb498a4da18e203456e09b59e7, role_id=bacea3b4b26e48a598b75ea1b301fafb)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:56.534100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a1223b-3eb9-4070-bb3b-31a31a9c2788 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:56.548654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 616/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:53:56 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/d3d7b3cb498a4da18e203456e09b59e7/roles/bacea3b4b26e48a598b75ea1b301fafb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:53:58.560699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b0935-9663-4f2e-b9ee-6224677012b7 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:58.561266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b0935-9663-4f2e-b9ee-6224677012b7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:58.561266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b0935-9663-4f2e-b9ee-6224677012b7 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:58.562586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 617/1233] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:53:58.568534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db2a0-0a43-4dec-910a-37f3a2c46a63 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:58.568986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db2a0-0a43-4dec-910a-37f3a2c46a63 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:58.569077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db2a0-0a43-4dec-910a-37f3a2c46a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:58.591974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6f7db2a0-0a43-4dec-910a-37f3a2c46a63 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:53:58.593349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6f7db2a0-0a43-4dec-910a-37f3a2c46a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:53:58.664520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 617/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:53:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:53:58.687650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b0935-9663-4f2e-b9ee-6224677012b7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:58.715843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:58.797863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b0935-9663-4f2e-b9ee-6224677012b7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:58.835512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d782aa-4853-4c44-b1ac-b9b3b2a77765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:58.836182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d782aa-4853-4c44-b1ac-b9b3b2a77765 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:58.836456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d782aa-4853-4c44-b1ac-b9b3b2a77765 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:58.836723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d782aa-4853-4c44-b1ac-b9b3b2a77765 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:58.848887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d782aa-4853-4c44-b1ac-b9b3b2a77765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-73d782aa-4853-4c44-b1ac-b9b3b2a77765 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:53:58.854027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:58.855149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:53:58.860986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 618/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 10 09:53:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:58.861800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db2a0-0a43-4dec-910a-37f3a2c46a63 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:58.915594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:58.972205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db2a0-0a43-4dec-910a-37f3a2c46a63 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:59.007188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517025a-bf00-4d5d-8d12-76c21f7ccc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:59.007792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517025a-bf00-4d5d-8d12-76c21f7ccc10 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:59.008068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517025a-bf00-4d5d-8d12-76c21f7ccc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:59.008337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517025a-bf00-4d5d-8d12-76c21f7ccc10 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:59.008991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517025a-bf00-4d5d-8d12-76c21f7ccc10 None admin] RBAC: Adding query filter params (name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:59.009289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517025a-bf00-4d5d-8d12-76c21f7ccc10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:59.010646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517025a-bf00-4d5d-8d12-76c21f7ccc10 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:59.018652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 618/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 10 09:53:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:53:59.025136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d782aa-4853-4c44-b1ac-b9b3b2a77765 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:59.111214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:59.127253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d782aa-4853-4c44-b1ac-b9b3b2a77765 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:59.164621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bc870-5817-4054-b105-455b60c7f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:59.165248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bc870-5817-4054-b105-455b60c7f390 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:59.165537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bc870-5817-4054-b105-455b60c7f390 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:59.166012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bc870-5817-4054-b105-455b60c7f390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:59.176593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bc870-5817-4054-b105-455b60c7f390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:59.178065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bc870-5817-4054-b105-455b60c7f390 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-033bc870-5817-4054-b105-455b60c7f390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:53:59.198050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:59.199251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:53:59.200900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:53:59.200900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:53:59.200900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:53:59.200900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 619/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:59.205973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517025a-bf00-4d5d-8d12-76c21f7ccc10 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:59.308519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517025a-bf00-4d5d-8d12-76c21f7ccc10 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:59.317987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:59.343470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e62e1-486e-4bfd-bbd0-969e13f11fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:59.344089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e62e1-486e-4bfd-bbd0-969e13f11fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:59.344361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e62e1-486e-4bfd-bbd0-969e13f11fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:59.344626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e62e1-486e-4bfd-bbd0-969e13f11fd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:59.345268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e62e1-486e-4bfd-bbd0-969e13f11fd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:59.345563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e62e1-486e-4bfd-bbd0-969e13f11fd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:59.347156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e62e1-486e-4bfd-bbd0-969e13f11fd3 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:59.359830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 619/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:59.366648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bc870-5817-4054-b105-455b60c7f390 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:59.475610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bc870-5817-4054-b105-455b60c7f390 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:59.510697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe04ec-0100-41fb-8d94-6768651231eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:59.511334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe04ec-0100-41fb-8d94-6768651231eb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:59.511615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe04ec-0100-41fb-8d94-6768651231eb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:59.511887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe04ec-0100-41fb-8d94-6768651231eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:59.514578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:59.519754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe04ec-0100-41fb-8d94-6768651231eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:59.521365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe04ec-0100-41fb-8d94-6768651231eb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0ebe04ec-0100-41fb-8d94-6768651231eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:53:59.532470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:53:59.538800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:53:59.540001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 620/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:53:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:53:59.540866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e62e1-486e-4bfd-bbd0-969e13f11fd3 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:59.645570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e62e1-486e-4bfd-bbd0-969e13f11fd3 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:59.683410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860892-d354-4d34-b869-d090afba9ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:59.684025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860892-d354-4d34-b869-d090afba9ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:59.684301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860892-d354-4d34-b869-d090afba9ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:59.684571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860892-d354-4d34-b869-d090afba9ab4 None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:59.685296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860892-d354-4d34-b869-d090afba9ab4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:53:59.685597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860892-d354-4d34-b869-d090afba9ab4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:59.687050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860892-d354-4d34-b869-d090afba9ab4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:53:59.705044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 620/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 10 09:53:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:53:59.711323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe04ec-0100-41fb-8d94-6768651231eb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:53:59.822525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe04ec-0100-41fb-8d94-6768651231eb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:53:59.857759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b993b0-465d-48fe-99f4-78a6212f0bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:53:59.858365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b993b0-465d-48fe-99f4-78a6212f0bac None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:53:59.858643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b993b0-465d-48fe-99f4-78a6212f0bac None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:53:59.858910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b993b0-465d-48fe-99f4-78a6212f0bac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:53:59.923695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:53:59.931510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b993b0-465d-48fe-99f4-78a6212f0bac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:53:59.932957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b993b0-465d-48fe-99f4-78a6212f0bac None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:00.037370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 621/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:53:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:54:00.043591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860892-d354-4d34-b869-d090afba9ab4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:00.074333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:00.155336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860892-d354-4d34-b869-d090afba9ab4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:00.200117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a78527-c740-42ea-82ab-8bb3b98530f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:00.200724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a78527-c740-42ea-82ab-8bb3b98530f0 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:00.201002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a78527-c740-42ea-82ab-8bb3b98530f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:00.201270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a78527-c740-42ea-82ab-8bb3b98530f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:00.211845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a78527-c740-42ea-82ab-8bb3b98530f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:00.213280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a78527-c740-42ea-82ab-8bb3b98530f0 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-d6a78527-c740-42ea-82ab-8bb3b98530f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:54:00.228526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:00.229703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:54:00.230771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:54:00.230771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:00.230771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:00.230771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 621/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:00.235862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b993b0-465d-48fe-99f4-78a6212f0bac None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:00.334903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:00.341971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b993b0-465d-48fe-99f4-78a6212f0bac None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:00.379222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a81527-3812-4bea-801b-a5585f7e65d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:00.379936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a81527-3812-4bea-801b-a5585f7e65d0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:00.380253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a81527-3812-4bea-801b-a5585f7e65d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:00.380584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a81527-3812-4bea-801b-a5585f7e65d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:00.381217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a81527-3812-4bea-801b-a5585f7e65d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:00.381516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a81527-3812-4bea-801b-a5585f7e65d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:00.383139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a81527-3812-4bea-801b-a5585f7e65d0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:00.395158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 622/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:00.401513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a78527-c740-42ea-82ab-8bb3b98530f0 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:00.483663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:00.506735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a78527-c740-42ea-82ab-8bb3b98530f0 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:00.542257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc1be7-640b-4228-85ec-df1c5d429b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:00.542907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc1be7-640b-4228-85ec-df1c5d429b8a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:00.543217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc1be7-640b-4228-85ec-df1c5d429b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:00.543489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc1be7-640b-4228-85ec-df1c5d429b8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:00.550301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc1be7-640b-4228-85ec-df1c5d429b8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:00.551794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc1be7-640b-4228-85ec-df1c5d429b8a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-dedc1be7-640b-4228-85ec-df1c5d429b8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:54:00.563135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:00.564383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:00.566065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:54:00.566065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:54:00.566065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:54:00.566065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:54:00.566065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:54:00.566065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:54:00.566065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:54:00.566065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:00.566065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 622/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:00.571201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a81527-3812-4bea-801b-a5585f7e65d0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:00.673324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a81527-3812-4bea-801b-a5585f7e65d0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:00.709909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a374e3-0183-455c-a01f-7ff2409ea308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:00.710532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a374e3-0183-455c-a01f-7ff2409ea308 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:00.710808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a374e3-0183-455c-a01f-7ff2409ea308 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:00.711190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a374e3-0183-455c-a01f-7ff2409ea308 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:00.711922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a374e3-0183-455c-a01f-7ff2409ea308 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:00.712548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a374e3-0183-455c-a01f-7ff2409ea308 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:00.713670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a374e3-0183-455c-a01f-7ff2409ea308 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:00.726875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 623/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:00.733053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc1be7-640b-4228-85ec-df1c5d429b8a None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:00.844589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc1be7-640b-4228-85ec-df1c5d429b8a None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:00.882110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed930b-0f05-47f6-ab8c-b9f124c2f5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:00.882837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed930b-0f05-47f6-ab8c-b9f124c2f5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:00.883154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed930b-0f05-47f6-ab8c-b9f124c2f5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:00.883480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed930b-0f05-47f6-ab8c-b9f124c2f5e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:00.884294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed930b-0f05-47f6-ab8c-b9f124c2f5e5 None admin] RBAC: Adding query filter params (role.id=bacea3b4b26e48a598b75ea1b301fafb, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=d3d7b3cb498a4da18e203456e09b59e7) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:00.884658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed930b-0f05-47f6-ab8c-b9f124c2f5e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:00.886169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed930b-0f05-47f6-ab8c-b9f124c2f5e5 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:00.892488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 623/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:54:00 2022] GET /identity/v3/role_assignments?user.id=d3d7b3cb498a4da18e203456e09b59e7&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=bacea3b4b26e48a598b75ea1b301fafb => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:02.943359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81380ec-fac6-436d-84e7-4460d9f4be2e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:02.943885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81380ec-fac6-436d-84e7-4460d9f4be2e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:02.943885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81380ec-fac6-436d-84e7-4460d9f4be2e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:02.945247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 624/1247] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:54:02.950688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125682dc-5bc4-47a8-a503-9e5fc92adde4 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:02.951257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125682dc-5bc4-47a8-a503-9e5fc92adde4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:02.951257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125682dc-5bc4-47a8-a503-9e5fc92adde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:02.959479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:02.972842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-125682dc-5bc4-47a8-a503-9e5fc92adde4 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:54:02.974029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-125682dc-5bc4-47a8-a503-9e5fc92adde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:54:03.046393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 624/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:54:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:54:03.070525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81380ec-fac6-436d-84e7-4460d9f4be2e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:03.089770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:03.180429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81380ec-fac6-436d-84e7-4460d9f4be2e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:03.217546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280929e0-dc77-45f2-9b8a-d9581d3b8150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:03.218100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280929e0-dc77-45f2-9b8a-d9581d3b8150 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:03.218375 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280929e0-dc77-45f2-9b8a-d9581d3b8150 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:03.218641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280929e0-dc77-45f2-9b8a-d9581d3b8150 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:03.229420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280929e0-dc77-45f2-9b8a-d9581d3b8150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-280929e0-dc77-45f2-9b8a-d9581d3b8150 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:54:03.233966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:03.235139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:03.236034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 625/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:54:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:54:03.241576 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125682dc-5bc4-47a8-a503-9e5fc92adde4 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:03.350199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125682dc-5bc4-47a8-a503-9e5fc92adde4 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:03.382174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:03.387513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733f406-f136-4e28-87ef-b327fec8ac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:03.388117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733f406-f136-4e28-87ef-b327fec8ac52 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:03.388394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733f406-f136-4e28-87ef-b327fec8ac52 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:03.388670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733f406-f136-4e28-87ef-b327fec8ac52 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:03.389335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733f406-f136-4e28-87ef-b327fec8ac52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:03.389651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733f406-f136-4e28-87ef-b327fec8ac52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:03.391039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733f406-f136-4e28-87ef-b327fec8ac52 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:03.401557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 625/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:54:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:03.407520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280929e0-dc77-45f2-9b8a-d9581d3b8150 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:03.495922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:03.520738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280929e0-dc77-45f2-9b8a-d9581d3b8150 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:03.558221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2de34-c7a6-4069-91db-023eb39dd81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:03.558851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2de34-c7a6-4069-91db-023eb39dd81e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:03.559155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2de34-c7a6-4069-91db-023eb39dd81e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:03.559428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2de34-c7a6-4069-91db-023eb39dd81e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:03.569940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2de34-c7a6-4069-91db-023eb39dd81e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:03.571423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2de34-c7a6-4069-91db-023eb39dd81e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6fb2de34-c7a6-4069-91db-023eb39dd81e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:54:03.588290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:03.589344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:54:03.590420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:54:03.590420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:03.590420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:03.590420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 626/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:54:03.596249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733f406-f136-4e28-87ef-b327fec8ac52 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:03.709367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733f406-f136-4e28-87ef-b327fec8ac52 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:03.748183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b67542-11e3-4977-9e00-00fbb35cac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:03.748719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b67542-11e3-4977-9e00-00fbb35cac3d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:03.748990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b67542-11e3-4977-9e00-00fbb35cac3d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:03.749256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b67542-11e3-4977-9e00-00fbb35cac3d None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:03.749888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b67542-11e3-4977-9e00-00fbb35cac3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:03.750181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b67542-11e3-4977-9e00-00fbb35cac3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:03.751601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b67542-11e3-4977-9e00-00fbb35cac3d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:03.764144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 626/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:03.770781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2de34-c7a6-4069-91db-023eb39dd81e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:03.883464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2de34-c7a6-4069-91db-023eb39dd81e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:03.909809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:03.921364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c14e6-efc6-48cc-bdf3-0a6c20d61901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:03.922123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c14e6-efc6-48cc-bdf3-0a6c20d61901 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:03.922407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c14e6-efc6-48cc-bdf3-0a6c20d61901 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:03.922680 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c14e6-efc6-48cc-bdf3-0a6c20d61901 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:03.931139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c14e6-efc6-48cc-bdf3-0a6c20d61901 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:03.932586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c14e6-efc6-48cc-bdf3-0a6c20d61901 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9c1c14e6-efc6-48cc-bdf3-0a6c20d61901 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:54:03.946620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:03.948153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:03.949418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 627/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:03.954254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b67542-11e3-4977-9e00-00fbb35cac3d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:04.014091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:04.071166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b67542-11e3-4977-9e00-00fbb35cac3d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:04.110239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fc2a1-806e-430b-a3bd-966cc38b855d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:04.110919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fc2a1-806e-430b-a3bd-966cc38b855d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:04.111225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fc2a1-806e-430b-a3bd-966cc38b855d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:04.111491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fc2a1-806e-430b-a3bd-966cc38b855d None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:04.112829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fc2a1-806e-430b-a3bd-966cc38b855d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:04.113133 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fc2a1-806e-430b-a3bd-966cc38b855d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:04.114547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fc2a1-806e-430b-a3bd-966cc38b855d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:04.134581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 627/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 10 09:54:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:04.140454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c14e6-efc6-48cc-bdf3-0a6c20d61901 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:04.253028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c14e6-efc6-48cc-bdf3-0a6c20d61901 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:04.288334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29691f-1184-42f6-b22f-963db0f65b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:04.288901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29691f-1184-42f6-b22f-963db0f65b0a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:04.289276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29691f-1184-42f6-b22f-963db0f65b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:04.289588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29691f-1184-42f6-b22f-963db0f65b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:04.351874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:04.368774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29691f-1184-42f6-b22f-963db0f65b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:04.370970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29691f-1184-42f6-b22f-963db0f65b0a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:04.487147 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 628/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:54:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:54:04.493327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fc2a1-806e-430b-a3bd-966cc38b855d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:04.602552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fc2a1-806e-430b-a3bd-966cc38b855d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:04.616749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:04.637933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f5df8-489b-44a7-84e4-1f83025cde8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:04.638561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f5df8-489b-44a7-84e4-1f83025cde8c None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:04.638855 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f5df8-489b-44a7-84e4-1f83025cde8c None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:04.639141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f5df8-489b-44a7-84e4-1f83025cde8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:04.651178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f5df8-489b-44a7-84e4-1f83025cde8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:04.652678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f5df8-489b-44a7-84e4-1f83025cde8c None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-2b4f5df8-489b-44a7-84e4-1f83025cde8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:54:04.668089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:04.669216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:54:04.670116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:54:04.670116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:04.670116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:04.670116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 628/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:04.675114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29691f-1184-42f6-b22f-963db0f65b0a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:04.762024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:04.776002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29691f-1184-42f6-b22f-963db0f65b0a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:04.811762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57257aab-b421-4398-b1d0-89ba94081772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:04.812368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57257aab-b421-4398-b1d0-89ba94081772 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:04.812643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57257aab-b421-4398-b1d0-89ba94081772 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:04.812915 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57257aab-b421-4398-b1d0-89ba94081772 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:04.813549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57257aab-b421-4398-b1d0-89ba94081772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:04.813848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57257aab-b421-4398-b1d0-89ba94081772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:04.815250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57257aab-b421-4398-b1d0-89ba94081772 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:04.827149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 629/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:04.832601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f5df8-489b-44a7-84e4-1f83025cde8c None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:04.942806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f5df8-489b-44a7-84e4-1f83025cde8c None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:04.977468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cede9d-2b9c-43a3-bd3f-f9c933a690ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:04.978117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cede9d-2b9c-43a3-bd3f-f9c933a690ae None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:04.978363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cede9d-2b9c-43a3-bd3f-f9c933a690ae None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:04.978634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cede9d-2b9c-43a3-bd3f-f9c933a690ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:04.985359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cede9d-2b9c-43a3-bd3f-f9c933a690ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:04.986812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cede9d-2b9c-43a3-bd3f-f9c933a690ae None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-59cede9d-2b9c-43a3-bd3f-f9c933a690ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:54:04.998063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:04.999136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:05.000115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:54:05.000115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:54:05.000115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:54:05.000115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:54:05.000115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:54:05.000115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:54:05.000115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:54:05.000115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:05.000115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 629/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:54:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 09:54:05.006193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57257aab-b421-4398-b1d0-89ba94081772 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:05.109473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57257aab-b421-4398-b1d0-89ba94081772 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:05.145642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d589ea-c326-486a-9e72-88ef89e74ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:05.146577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d589ea-c326-486a-9e72-88ef89e74ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:05.147227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d589ea-c326-486a-9e72-88ef89e74ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:05.147602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d589ea-c326-486a-9e72-88ef89e74ab0 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:05.148607 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d589ea-c326-486a-9e72-88ef89e74ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:05.149069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d589ea-c326-486a-9e72-88ef89e74ab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:05.151243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d589ea-c326-486a-9e72-88ef89e74ab0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:05.167765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 630/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:05.174093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cede9d-2b9c-43a3-bd3f-f9c933a690ae None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:05.205353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:05.284054 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cede9d-2b9c-43a3-bd3f-f9c933a690ae None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:05.320144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf88878-17a5-41af-88dd-30c2adbf563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:05.320820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf88878-17a5-41af-88dd-30c2adbf563e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:05.321034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf88878-17a5-41af-88dd-30c2adbf563e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:05.321320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf88878-17a5-41af-88dd-30c2adbf563e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:05.322010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf88878-17a5-41af-88dd-30c2adbf563e None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=d3d7b3cb498a4da18e203456e09b59e7) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:05.322317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf88878-17a5-41af-88dd-30c2adbf563e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:05.323763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf88878-17a5-41af-88dd-30c2adbf563e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:05.328954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 630/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:54:05 2022] GET /identity/v3/role_assignments?user.id=d3d7b3cb498a4da18e203456e09b59e7&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=8417343e9ef64991bcf51bb9692da102 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:07.319726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5df170-c954-44ce-a3aa-195a65c153ff None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:07.320251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5df170-c954-44ce-a3aa-195a65c153ff None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:07.320251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5df170-c954-44ce-a3aa-195a65c153ff None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:07.321605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 631/1261] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:54:07.326697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bde94-0759-410e-b89b-08845867f914 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:07.327150 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bde94-0759-410e-b89b-08845867f914 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:07.327265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bde94-0759-410e-b89b-08845867f914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:07.345261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e93bde94-0759-410e-b89b-08845867f914 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:54:07.346500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e93bde94-0759-410e-b89b-08845867f914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:54:07.410335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 631/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:54:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:54:07.431826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5df170-c954-44ce-a3aa-195a65c153ff None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:07.443101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:07.530880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5df170-c954-44ce-a3aa-195a65c153ff None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:07.563799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba561d-d4ce-486f-9588-96b53616b6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:07.564409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba561d-d4ce-486f-9588-96b53616b6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:07.564689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba561d-d4ce-486f-9588-96b53616b6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:07.564963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba561d-d4ce-486f-9588-96b53616b6a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:07.575406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba561d-d4ce-486f-9588-96b53616b6a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-55ba561d-d4ce-486f-9588-96b53616b6a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:54:07.579909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:07.581014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:07.582201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 632/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:54:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:07.587052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bde94-0759-410e-b89b-08845867f914 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:07.604384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:07.680752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bde94-0759-410e-b89b-08845867f914 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:07.712778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a2841-9b5a-43d9-8521-f5950efa79e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:07.713362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a2841-9b5a-43d9-8521-f5950efa79e2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:07.713652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a2841-9b5a-43d9-8521-f5950efa79e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:07.713904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a2841-9b5a-43d9-8521-f5950efa79e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:07.714551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a2841-9b5a-43d9-8521-f5950efa79e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:07.714848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a2841-9b5a-43d9-8521-f5950efa79e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:07.716221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a2841-9b5a-43d9-8521-f5950efa79e2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:07.723844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 632/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:54:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:07.729532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba561d-d4ce-486f-9588-96b53616b6a1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:07.801104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:07.833882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba561d-d4ce-486f-9588-96b53616b6a1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:07.866513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b352b-ed22-4597-ae64-9594fb59c5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:07.867155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b352b-ed22-4597-ae64-9594fb59c5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:07.867437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b352b-ed22-4597-ae64-9594fb59c5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:07.867708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b352b-ed22-4597-ae64-9594fb59c5d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:07.877503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b352b-ed22-4597-ae64-9594fb59c5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:07.879630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b352b-ed22-4597-ae64-9594fb59c5d6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-534b352b-ed22-4597-ae64-9594fb59c5d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:54:07.893441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:07.894524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:54:07.895631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:54:07.895631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:07.895631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:07.895631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 633/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:07.900409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a2841-9b5a-43d9-8521-f5950efa79e2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:07.978000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:07.996936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a2841-9b5a-43d9-8521-f5950efa79e2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:08.028062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240032f-81b2-45fb-bbfd-aebb056245c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:08.028654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240032f-81b2-45fb-bbfd-aebb056245c1 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:08.028962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240032f-81b2-45fb-bbfd-aebb056245c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:08.029226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240032f-81b2-45fb-bbfd-aebb056245c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:08.029850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240032f-81b2-45fb-bbfd-aebb056245c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:08.030148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240032f-81b2-45fb-bbfd-aebb056245c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:08.031524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240032f-81b2-45fb-bbfd-aebb056245c1 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:08.043039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 633/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:08.048646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b352b-ed22-4597-ae64-9594fb59c5d6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:08.148407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b352b-ed22-4597-ae64-9594fb59c5d6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:08.168815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:08.186188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ce4d1-4534-44bf-b7cb-f126992a5c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:08.186924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ce4d1-4534-44bf-b7cb-f126992a5c4d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:08.187325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ce4d1-4534-44bf-b7cb-f126992a5c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:08.187651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ce4d1-4534-44bf-b7cb-f126992a5c4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:08.195863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ce4d1-4534-44bf-b7cb-f126992a5c4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:08.197347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ce4d1-4534-44bf-b7cb-f126992a5c4d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-cd5ce4d1-4534-44bf-b7cb-f126992a5c4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:54:08.209235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:08.210477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:08.211910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 634/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:08.216384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240032f-81b2-45fb-bbfd-aebb056245c1 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:08.310881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240032f-81b2-45fb-bbfd-aebb056245c1 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:08.342919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c175a1-6e26-4c55-b6c3-bbdc3c874aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:08.343540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c175a1-6e26-4c55-b6c3-bbdc3c874aab None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:08.343813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c175a1-6e26-4c55-b6c3-bbdc3c874aab None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:08.344079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c175a1-6e26-4c55-b6c3-bbdc3c874aab None admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:08.344778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c175a1-6e26-4c55-b6c3-bbdc3c874aab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:08.345076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c175a1-6e26-4c55-b6c3-bbdc3c874aab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:08.347061 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c175a1-6e26-4c55-b6c3-bbdc3c874aab None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:08.355254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:08.363380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 634/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 10 09:54:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:08.369340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ce4d1-4534-44bf-b7cb-f126992a5c4d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:08.469275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ce4d1-4534-44bf-b7cb-f126992a5c4d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:08.505836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb316-041e-42c8-9a6d-cf3ca76b1ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:08.506454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb316-041e-42c8-9a6d-cf3ca76b1ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:08.506728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb316-041e-42c8-9a6d-cf3ca76b1ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:08.507019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb316-041e-42c8-9a6d-cf3ca76b1ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:08.516956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cb316-041e-42c8-9a6d-cf3ca76b1ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:08.518924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cb316-041e-42c8-9a6d-cf3ca76b1ab7 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f83cb316-041e-42c8-9a6d-cf3ca76b1ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:54:08.532216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:08.533191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:54:08.534785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:54:08.534785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:08.534785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:08.534785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 635/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:08.539720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c175a1-6e26-4c55-b6c3-bbdc3c874aab None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:08.634833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c175a1-6e26-4c55-b6c3-bbdc3c874aab None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:08.668418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bc9e0-1081-40a1-a61a-2e1ddfd161c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:08.668955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bc9e0-1081-40a1-a61a-2e1ddfd161c6 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:08.669229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bc9e0-1081-40a1-a61a-2e1ddfd161c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:08.669494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bc9e0-1081-40a1-a61a-2e1ddfd161c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:08.670114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bc9e0-1081-40a1-a61a-2e1ddfd161c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:08.670408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bc9e0-1081-40a1-a61a-2e1ddfd161c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:08.671783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bc9e0-1081-40a1-a61a-2e1ddfd161c6 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:08.682331 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 635/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:08.687864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb316-041e-42c8-9a6d-cf3ca76b1ab7 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:08.720278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:08.786624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb316-041e-42c8-9a6d-cf3ca76b1ab7 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:08.822267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e0e17-1303-4384-b7ba-e86cc5d4ba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:08.822888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e0e17-1303-4384-b7ba-e86cc5d4ba0b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:08.823193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e0e17-1303-4384-b7ba-e86cc5d4ba0b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:08.823465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e0e17-1303-4384-b7ba-e86cc5d4ba0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:08.829751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e0e17-1303-4384-b7ba-e86cc5d4ba0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:08.831806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e0e17-1303-4384-b7ba-e86cc5d4ba0b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-963e0e17-1303-4384-b7ba-e86cc5d4ba0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:54:08.843316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:08.844353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:08.845313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:54:08.845313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:54:08.845313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:54:08.845313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:54:08.845313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:54:08.845313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:54:08.845313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:54:08.845313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:08.845313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 636/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:54:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:08.850224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bc9e0-1081-40a1-a61a-2e1ddfd161c6 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:08.905853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:08.945379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bc9e0-1081-40a1-a61a-2e1ddfd161c6 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:08.976998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e22efc-0b7d-4c2f-be89-be63ad76180a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:08.977592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e22efc-0b7d-4c2f-be89-be63ad76180a None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:08.977868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e22efc-0b7d-4c2f-be89-be63ad76180a None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:08.978136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e22efc-0b7d-4c2f-be89-be63ad76180a None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:08.978813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e22efc-0b7d-4c2f-be89-be63ad76180a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:08.979141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e22efc-0b7d-4c2f-be89-be63ad76180a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:08.980544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e22efc-0b7d-4c2f-be89-be63ad76180a None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:08.991777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 636/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:08.997808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e0e17-1303-4384-b7ba-e86cc5d4ba0b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:09.087201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:09.096131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e0e17-1303-4384-b7ba-e86cc5d4ba0b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:09.130059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce36cd-3861-4f48-b73d-ccaa25c7a1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:09.130697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce36cd-3861-4f48-b73d-ccaa25c7a1ff None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:09.130994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce36cd-3861-4f48-b73d-ccaa25c7a1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:09.131265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce36cd-3861-4f48-b73d-ccaa25c7a1ff None admin] PATH_INFO: `/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/d3d7b3cb498a4da18e203456e09b59e7/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:09.133100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce36cd-3861-4f48-b73d-ccaa25c7a1ff None admin] RBAC: Authorizing `identity:create_grant(project_id=d8afc1639a0c4c6c86a1d86d5c254513, user_id=d3d7b3cb498a4da18e203456e09b59e7, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:09.134520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce36cd-3861-4f48-b73d-ccaa25c7a1ff None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:09.146825 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 637/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 10 09:54:08 2022] PUT /identity/v3/projects/d8afc1639a0c4c6c86a1d86d5c254513/users/d3d7b3cb498a4da18e203456e09b59e7/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:54:11.139928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb61ed-c240-4b5d-b5f3-10cd13d0a82b None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:11.140496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb61ed-c240-4b5d-b5f3-10cd13d0a82b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:11.140496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb61ed-c240-4b5d-b5f3-10cd13d0a82b None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:11.141767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 637/1274] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:54:11.147400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bce03f-8f65-453d-a451-61544c94e61c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:11.147983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bce03f-8f65-453d-a451-61544c94e61c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:11.147983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bce03f-8f65-453d-a451-61544c94e61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:11.165537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-31bce03f-8f65-453d-a451-61544c94e61c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:54:11.166762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-31bce03f-8f65-453d-a451-61544c94e61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:54:11.230579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 638/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:54:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:54:11.254327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb61ed-c240-4b5d-b5f3-10cd13d0a82b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:11.350772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb61ed-c240-4b5d-b5f3-10cd13d0a82b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:11.367471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:11.386213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b799d5-72a7-43d1-916f-63009bd6e78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:11.386835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b799d5-72a7-43d1-916f-63009bd6e78f None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:11.387147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b799d5-72a7-43d1-916f-63009bd6e78f None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:11.387601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b799d5-72a7-43d1-916f-63009bd6e78f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:11.397884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b799d5-72a7-43d1-916f-63009bd6e78f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-46b799d5-72a7-43d1-916f-63009bd6e78f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 09:54:11.402361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:11.403497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:11.404664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 638/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 10 09:54:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:11.409418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bce03f-8f65-453d-a451-61544c94e61c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:11.485260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:11.504023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bce03f-8f65-453d-a451-61544c94e61c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:11.535388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38374f-c08d-4aef-824f-6ea142434214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:11.535973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38374f-c08d-4aef-824f-6ea142434214 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:11.536245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38374f-c08d-4aef-824f-6ea142434214 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:11.536510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38374f-c08d-4aef-824f-6ea142434214 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:11.537156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38374f-c08d-4aef-824f-6ea142434214 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:11.537453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38374f-c08d-4aef-824f-6ea142434214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:11.538780 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38374f-c08d-4aef-824f-6ea142434214 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:11.546353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 639/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 10 09:54:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:11.552445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b799d5-72a7-43d1-916f-63009bd6e78f None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:11.647327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b799d5-72a7-43d1-916f-63009bd6e78f None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:11.679298 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c08a3-8774-4df5-b161-9b92b18208b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:11.679850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c08a3-8774-4df5-b161-9b92b18208b2 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:11.680125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c08a3-8774-4df5-b161-9b92b18208b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:11.680391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c08a3-8774-4df5-b161-9b92b18208b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:11.689867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c08a3-8774-4df5-b161-9b92b18208b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:11.691925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c08a3-8774-4df5-b161-9b92b18208b2 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-5d0c08a3-8774-4df5-b161-9b92b18208b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:54:11.705529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:11.706589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:54:11.707846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:54:11.707846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:11.707846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:11.707846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 639/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:11.712099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38374f-c08d-4aef-824f-6ea142434214 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:11.806377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38374f-c08d-4aef-824f-6ea142434214 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:11.837201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0514a-90bd-454e-8def-fd667f6b35b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:11.837784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0514a-90bd-454e-8def-fd667f6b35b0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:11.838058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0514a-90bd-454e-8def-fd667f6b35b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:11.838324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0514a-90bd-454e-8def-fd667f6b35b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:11.838943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0514a-90bd-454e-8def-fd667f6b35b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:11.839265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0514a-90bd-454e-8def-fd667f6b35b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:11.840587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0514a-90bd-454e-8def-fd667f6b35b0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:11.852048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 640/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:11.857380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c08a3-8774-4df5-b161-9b92b18208b2 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:11.887097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:11.952243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c08a3-8774-4df5-b161-9b92b18208b2 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:11.983652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd27cec-fa21-4cde-9880-dc376093bc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:11.984248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd27cec-fa21-4cde-9880-dc376093bc71 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:11.984519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd27cec-fa21-4cde-9880-dc376093bc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:11.984790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd27cec-fa21-4cde-9880-dc376093bc71 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:11.993144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd27cec-fa21-4cde-9880-dc376093bc71 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:11.994836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd27cec-fa21-4cde-9880-dc376093bc71 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-8fd27cec-fa21-4cde-9880-dc376093bc71 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 09:54:12.005823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:12.006932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:12.008228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 640/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:12.012573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0514a-90bd-454e-8def-fd667f6b35b0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:12.016268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:12.108673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0514a-90bd-454e-8def-fd667f6b35b0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:12.140589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71096411-ab9b-405d-9543-9082d1f84de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:12.141181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71096411-ab9b-405d-9543-9082d1f84de6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:12.141458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71096411-ab9b-405d-9543-9082d1f84de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:12.141729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71096411-ab9b-405d-9543-9082d1f84de6 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:12.142443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71096411-ab9b-405d-9543-9082d1f84de6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:12.142743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71096411-ab9b-405d-9543-9082d1f84de6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:12.144697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71096411-ab9b-405d-9543-9082d1f84de6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:12.166476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 641/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 10 09:54:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:12.172384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd27cec-fa21-4cde-9880-dc376093bc71 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:12.257961 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:12.267247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd27cec-fa21-4cde-9880-dc376093bc71 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:12.299439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0591200-6239-4883-951b-f56a13e217ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:12.300020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0591200-6239-4883-951b-f56a13e217ea None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:12.300294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0591200-6239-4883-951b-f56a13e217ea None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:12.300561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0591200-6239-4883-951b-f56a13e217ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:12.363058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0591200-6239-4883-951b-f56a13e217ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:12.364468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0591200-6239-4883-951b-f56a13e217ea None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:12.447963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:12.458391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 641/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 10 09:54:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 09:54:12.464340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71096411-ab9b-405d-9543-9082d1f84de6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:12.523025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:12.559337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71096411-ab9b-405d-9543-9082d1f84de6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:12.591984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63ba6-030d-4a8c-9ca8-77a1234fb5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:12.592590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63ba6-030d-4a8c-9ca8-77a1234fb5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:12.592864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63ba6-030d-4a8c-9ca8-77a1234fb5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:12.593132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63ba6-030d-4a8c-9ca8-77a1234fb5c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:12.602619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b63ba6-030d-4a8c-9ca8-77a1234fb5c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:12.604076 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b63ba6-030d-4a8c-9ca8-77a1234fb5c3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c9b63ba6-030d-4a8c-9ca8-77a1234fb5c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 09:54:12.617726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:12.618836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 09:54:12.620342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 09:54:12.620342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 09:54:12.620342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:12.620342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 642/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:12.624573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0591200-6239-4883-951b-f56a13e217ea None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:12.718740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0591200-6239-4883-951b-f56a13e217ea None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:12.750644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440eb77-48b7-49eb-9468-2fbc1350686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:12.751258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440eb77-48b7-49eb-9468-2fbc1350686d None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:12.751537 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440eb77-48b7-49eb-9468-2fbc1350686d None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:12.751809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440eb77-48b7-49eb-9468-2fbc1350686d None admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:12.752434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440eb77-48b7-49eb-9468-2fbc1350686d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:12.753259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440eb77-48b7-49eb-9468-2fbc1350686d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:12.754611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440eb77-48b7-49eb-9468-2fbc1350686d None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:12.765234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 642/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 10 09:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:12.771448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63ba6-030d-4a8c-9ca8-77a1234fb5c3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:12.866170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63ba6-030d-4a8c-9ca8-77a1234fb5c3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:12.879797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:12.899678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8e83a-ae86-49d8-9480-77dca65596d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:12.900280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8e83a-ae86-49d8-9480-77dca65596d8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:12.900560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8e83a-ae86-49d8-9480-77dca65596d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:12.900832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8e83a-ae86-49d8-9480-77dca65596d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:12.907133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8e83a-ae86-49d8-9480-77dca65596d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:12.908544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8e83a-ae86-49d8-9480-77dca65596d8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-80e8e83a-ae86-49d8-9480-77dca65596d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 09:54:12.919762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:12.920783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:12.921945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 09:54:12.921945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 09:54:12.921945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 09:54:12.921945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 09:54:12.921945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 09:54:12.921945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 09:54:12.921945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 09:54:12.921945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Aug 10 09:54:12.921945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 643/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 10 09:54:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:12.926451 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440eb77-48b7-49eb-9468-2fbc1350686d None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:12.974398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:13.023320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440eb77-48b7-49eb-9468-2fbc1350686d None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:13.055278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215acf41-2d63-4e0c-8b20-d471ea20133e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:13.055865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215acf41-2d63-4e0c-8b20-d471ea20133e None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:13.056140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215acf41-2d63-4e0c-8b20-d471ea20133e None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:13.056407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215acf41-2d63-4e0c-8b20-d471ea20133e None admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:13.057079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215acf41-2d63-4e0c-8b20-d471ea20133e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:13.057380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215acf41-2d63-4e0c-8b20-d471ea20133e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:13.058801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215acf41-2d63-4e0c-8b20-d471ea20133e None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:13.070150 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 643/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 10 09:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:13.076203 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8e83a-ae86-49d8-9480-77dca65596d8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:13.174413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8e83a-ae86-49d8-9480-77dca65596d8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:13.207251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d3672-1c98-4c72-adbe-79c9cb1fd7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:13.207842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d3672-1c98-4c72-adbe-79c9cb1fd7da None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:13.208121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d3672-1c98-4c72-adbe-79c9cb1fd7da None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:13.208394 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d3672-1c98-4c72-adbe-79c9cb1fd7da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:13.209105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d3672-1c98-4c72-adbe-79c9cb1fd7da None admin] RBAC: Adding query filter params (role.id=8417343e9ef64991bcf51bb9692da102, scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513, user.id=d3d7b3cb498a4da18e203456e09b59e7) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:13.209412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d3672-1c98-4c72-adbe-79c9cb1fd7da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:13.210814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d3672-1c98-4c72-adbe-79c9cb1fd7da None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:13.216329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 644/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 10 09:54:13 2022] GET /identity/v3/role_assignments?user.id=d3d7b3cb498a4da18e203456e09b59e7&scope.project.id=d8afc1639a0c4c6c86a1d86d5c254513&role.id=8417343e9ef64991bcf51bb9692da102 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:15.201448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472121a-ef13-4486-a51b-9c3e04342b0a None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:15.201981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472121a-ef13-4486-a51b-9c3e04342b0a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:15.201981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472121a-ef13-4486-a51b-9c3e04342b0a None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:15.203572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 644/1288] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:54:15.209191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff120-e6c9-4d36-96f6-7c12672472e6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:15.209665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff120-e6c9-4d36-96f6-7c12672472e6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:15.209769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff120-e6c9-4d36-96f6-7c12672472e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:15.227278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-94bff120-e6c9-4d36-96f6-7c12672472e6 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:54:15.228457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-94bff120-e6c9-4d36-96f6-7c12672472e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:54:15.257773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:15.293211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 645/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:54:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 09:54:15.314740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472121a-ef13-4486-a51b-9c3e04342b0a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:15.409963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472121a-ef13-4486-a51b-9c3e04342b0a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:15.420936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:15.444283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367338b7-7ceb-44ec-8e38-94db819837a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:15.444892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367338b7-7ceb-44ec-8e38-94db819837a8 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:15.445163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367338b7-7ceb-44ec-8e38-94db819837a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:15.445426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367338b7-7ceb-44ec-8e38-94db819837a8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:15.445983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367338b7-7ceb-44ec-8e38-94db819837a8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:15.447372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367338b7-7ceb-44ec-8e38-94db819837a8 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-367338b7-7ceb-44ec-8e38-94db819837a8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:15.455659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:15.456834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:15.457707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 645/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:54:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:15.462534 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff120-e6c9-4d36-96f6-7c12672472e6 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:15.557465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff120-e6c9-4d36-96f6-7c12672472e6 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:15.590019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b98bb-07f7-4329-9b6a-0a1c5e94ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:15.590613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b98bb-07f7-4329-9b6a-0a1c5e94ac16 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:15.590892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b98bb-07f7-4329-9b6a-0a1c5e94ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:15.591195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b98bb-07f7-4329-9b6a-0a1c5e94ac16 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:15.591819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b98bb-07f7-4329-9b6a-0a1c5e94ac16 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:15.592125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b98bb-07f7-4329-9b6a-0a1c5e94ac16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:15.593486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b98bb-07f7-4329-9b6a-0a1c5e94ac16 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:15.598596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 646/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 10 09:54:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:15.604177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367338b7-7ceb-44ec-8e38-94db819837a8 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:15.698905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367338b7-7ceb-44ec-8e38-94db819837a8 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:15.730318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fbbcf-d0e2-4490-a0c3-e11577b8ad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:15.730835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fbbcf-d0e2-4490-a0c3-e11577b8ad46 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:15.731135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fbbcf-d0e2-4490-a0c3-e11577b8ad46 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:15.731402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fbbcf-d0e2-4490-a0c3-e11577b8ad46 None admin] PATH_INFO: `/v3/services` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:15.732005 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272fbbcf-d0e2-4490-a0c3-e11577b8ad46 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:15.732313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272fbbcf-d0e2-4490-a0c3-e11577b8ad46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:15.733621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272fbbcf-d0e2-4490-a0c3-e11577b8ad46 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:15.738345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 646/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 10 09:54:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:17.695628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c34a3e-dd0e-4950-be35-21ca3bfa6e67 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:17.696136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c34a3e-dd0e-4950-be35-21ca3bfa6e67 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:17.696136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c34a3e-dd0e-4950-be35-21ca3bfa6e67 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:17.697417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 647/1293] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:54:17.702453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f17b1c-8d85-4e66-a0aa-41ed1f65551f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:17.702820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f17b1c-8d85-4e66-a0aa-41ed1f65551f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:17.703023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f17b1c-8d85-4e66-a0aa-41ed1f65551f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:17.721009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c6f17b1c-8d85-4e66-a0aa-41ed1f65551f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:54:17.722185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c6f17b1c-8d85-4e66-a0aa-41ed1f65551f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:54:17.750232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:17.786736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 647/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:54:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:54:17.808630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c34a3e-dd0e-4950-be35-21ca3bfa6e67 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:17.839208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:17.907437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c34a3e-dd0e-4950-be35-21ca3bfa6e67 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:17.939374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9837f4-ad1c-43d1-8f7b-16d1f4cf490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:17.940478 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9837f4-ad1c-43d1-8f7b-16d1f4cf490d None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:17.940759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9837f4-ad1c-43d1-8f7b-16d1f4cf490d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:17.941032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9837f4-ad1c-43d1-8f7b-16d1f4cf490d None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:17.941655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9837f4-ad1c-43d1-8f7b-16d1f4cf490d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:17.943022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9837f4-ad1c-43d1-8f7b-16d1f4cf490d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:17.957059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 648/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 10 09:54:17 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:54:19.970692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f397167-d34a-4950-a742-9278e6e6a1f5 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:19.971759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f397167-d34a-4950-a742-9278e6e6a1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:19.971759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f397167-d34a-4950-a742-9278e6e6a1f5 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:19.973100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 648/1296] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:54:19.978074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b30f2e-8f42-4dc1-9b5b-d7ce451cda88 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:19.978490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b30f2e-8f42-4dc1-9b5b-d7ce451cda88 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:19.978602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b30f2e-8f42-4dc1-9b5b-d7ce451cda88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:19.996873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-47b30f2e-8f42-4dc1-9b5b-d7ce451cda88 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:54:19.998038 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-47b30f2e-8f42-4dc1-9b5b-d7ce451cda88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:54:20.068254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 649/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:54:19 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:54:20.090651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f397167-d34a-4950-a742-9278e6e6a1f5 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:20.211958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f397167-d34a-4950-a742-9278e6e6a1f5 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:20.246599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15ef61-12b9-4e98-8b59-cc0b4f03d2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:20.247180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15ef61-12b9-4e98-8b59-cc0b4f03d2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:20.247458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15ef61-12b9-4e98-8b59-cc0b4f03d2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:20.247730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15ef61-12b9-4e98-8b59-cc0b4f03d2d4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:20.248302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15ef61-12b9-4e98-8b59-cc0b4f03d2d4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:20.249687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15ef61-12b9-4e98-8b59-cc0b4f03d2d4 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-7d15ef61-12b9-4e98-8b59-cc0b4f03d2d4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:20.257950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:20.259129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:20.259990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 649/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:54:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:20.265045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b30f2e-8f42-4dc1-9b5b-d7ce451cda88 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:20.282453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:20.359393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b30f2e-8f42-4dc1-9b5b-d7ce451cda88 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:20.390920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb0424-e7a2-4bfe-9f1e-e87904497af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:20.391529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb0424-e7a2-4bfe-9f1e-e87904497af2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:20.391802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb0424-e7a2-4bfe-9f1e-e87904497af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:20.392066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb0424-e7a2-4bfe-9f1e-e87904497af2 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:20.392678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb0424-e7a2-4bfe-9f1e-e87904497af2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:20.393505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb0424-e7a2-4bfe-9f1e-e87904497af2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:20.394836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb0424-e7a2-4bfe-9f1e-e87904497af2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:20.399969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 650/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 10 09:54:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:20.405552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15ef61-12b9-4e98-8b59-cc0b4f03d2d4 None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:20.436465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:20.502117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15ef61-12b9-4e98-8b59-cc0b4f03d2d4 None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:20.534520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fcec6-2d18-4075-91b7-2f51f8613bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:20.535133 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fcec6-2d18-4075-91b7-2f51f8613bea None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:20.535410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fcec6-2d18-4075-91b7-2f51f8613bea None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:20.535677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fcec6-2d18-4075-91b7-2f51f8613bea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:20.536365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fcec6-2d18-4075-91b7-2f51f8613bea None admin] RBAC: Adding query filter params (interface=public, service_id=32b418a5de504ff3b2c0d413268e1ca7, region_id=RegionOne) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:20.536669 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fcec6-2d18-4075-91b7-2f51f8613bea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:20.538016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fcec6-2d18-4075-91b7-2f51f8613bea None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:20.542787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 650/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 10 09:54:20 2022] GET /identity/v3/endpoints?service_id=32b418a5de504ff3b2c0d413268e1ca7&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:20.548448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb0424-e7a2-4bfe-9f1e-e87904497af2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:20.626022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:20.644236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb0424-e7a2-4bfe-9f1e-e87904497af2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:20.676398 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52598b4f-14eb-4017-bdf5-6fea4c382342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:20.676985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52598b4f-14eb-4017-bdf5-6fea4c382342 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:20.677260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52598b4f-14eb-4017-bdf5-6fea4c382342 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:20.677529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52598b4f-14eb-4017-bdf5-6fea4c382342 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:20.678118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52598b4f-14eb-4017-bdf5-6fea4c382342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:20.679497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52598b4f-14eb-4017-bdf5-6fea4c382342 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:20.686018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 651/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 10 09:54:20 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:54:22.619161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f14b0-20d6-4a57-a4a6-b0ee2a9f0af7 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:22.619703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f14b0-20d6-4a57-a4a6-b0ee2a9f0af7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:22.619703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f14b0-20d6-4a57-a4a6-b0ee2a9f0af7 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:22.621036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 651/1302] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:54:22.626159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57321f-37ea-48c5-8d6c-9c1831a2382b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:22.626544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57321f-37ea-48c5-8d6c-9c1831a2382b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:22.626710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57321f-37ea-48c5-8d6c-9c1831a2382b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:22.644738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6f57321f-37ea-48c5-8d6c-9c1831a2382b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:54:22.645904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6f57321f-37ea-48c5-8d6c-9c1831a2382b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:54:22.708521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 652/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:54:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 09:54:22.730477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f14b0-20d6-4a57-a4a6-b0ee2a9f0af7 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:22.822681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:22.829746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f14b0-20d6-4a57-a4a6-b0ee2a9f0af7 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:22.861264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8dd11-8a16-43a0-8bba-5d5d83e7d6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:22.862383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8dd11-8a16-43a0-8bba-5d5d83e7d6ab None admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:22.862663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8dd11-8a16-43a0-8bba-5d5d83e7d6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:22.862934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8dd11-8a16-43a0-8bba-5d5d83e7d6ab None admin] PATH_INFO: `/v3/services/placement` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:22.863533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8dd11-8a16-43a0-8bba-5d5d83e7d6ab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:22.864891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8dd11-8a16-43a0-8bba-5d5d83e7d6ab None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application [None req-2bc8dd11-8a16-43a0-8bba-5d5d83e7d6ab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 09:54:22.873474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 09:54:22.879954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application with Lock( Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self._enter() Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application return self.creator() Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application created_value = creator( Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: ERROR keystone.server.flask.application Aug 10 09:54:22.880842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 652/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 10 09:54:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 09:54:22.881652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57321f-37ea-48c5-8d6c-9c1831a2382b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:22.953111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:22.975715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57321f-37ea-48c5-8d6c-9c1831a2382b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:23.008934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67433861-adc2-40e9-b800-fd3668e629af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:23.009525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67433861-adc2-40e9-b800-fd3668e629af None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:23.009800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67433861-adc2-40e9-b800-fd3668e629af None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:23.010067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67433861-adc2-40e9-b800-fd3668e629af None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:23.010685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67433861-adc2-40e9-b800-fd3668e629af None admin] RBAC: Adding query filter params (name=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:54:23.011006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67433861-adc2-40e9-b800-fd3668e629af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:23.012343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67433861-adc2-40e9-b800-fd3668e629af None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:23.017224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 653/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 10 09:54:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:54:23.023108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8dd11-8a16-43a0-8bba-5d5d83e7d6ab None admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:54:23.119464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8dd11-8a16-43a0-8bba-5d5d83e7d6ab None admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:54:23.151925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6724f30-ea3b-4955-a8a9-578acfd4e141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:54:23.152520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6724f30-ea3b-4955-a8a9-578acfd4e141 None admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:54:23.152799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6724f30-ea3b-4955-a8a9-578acfd4e141 None admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:54:23.153069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6724f30-ea3b-4955-a8a9-578acfd4e141 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:54:23.153726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6724f30-ea3b-4955-a8a9-578acfd4e141 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:54:23.155108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6724f30-ea3b-4955-a8a9-578acfd4e141 None admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:54:23.175587 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 653/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:54:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:54:33.014312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=30 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:54:33.170458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:24.740231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fcbc9-d430-4fe2-a086-eff12c2dc2be None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:24.740787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fcbc9-d430-4fe2-a086-eff12c2dc2be None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:24.740787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fcbc9-d430-4fe2-a086-eff12c2dc2be None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:24.746083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 654/1307] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:24.747745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8f59f-0e1d-48dc-8fc0-cb5ac51aff53 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:24.748510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8f59f-0e1d-48dc-8fc0-cb5ac51aff53 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:24.748956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8f59f-0e1d-48dc-8fc0-cb5ac51aff53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:24.932394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-26d8f59f-0e1d-48dc-8fc0-cb5ac51aff53 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:24.936069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-26d8f59f-0e1d-48dc-8fc0-cb5ac51aff53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:25.090985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 654/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:25.136222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34032a09-418a-4aea-977e-4c7893808496 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:25.136506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34032a09-418a-4aea-977e-4c7893808496 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:25.136954 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34032a09-418a-4aea-977e-4c7893808496 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:25.138283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 655/1309] 10.222.0.20 () {54 vars in 965 bytes} [Wed Aug 10 09:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:25.144211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae2820-a554-490b-8d7a-a1fda1877e65 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:25.144503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae2820-a554-490b-8d7a-a1fda1877e65 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:25.144786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae2820-a554-490b-8d7a-a1fda1877e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:25.224858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-91ae2820-a554-490b-8d7a-a1fda1877e65 None None] MFA Rules not processed for user `ab9b7576a3834862af6abb7e0416f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:25.226096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-91ae2820-a554-490b-8d7a-a1fda1877e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:25.248240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:25.306490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 655/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Wed Aug 10 09:55:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:25.316128 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34032a09-418a-4aea-977e-4c7893808496 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:25.317877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-34032a09-418a-4aea-977e-4c7893808496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:25.440119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34032a09-418a-4aea-977e-4c7893808496 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:25.516271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b4929-53a4-4ed3-b43e-d487cf4e1833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:25.516864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b4929-53a4-4ed3-b43e-d487cf4e1833 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:25.517145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b4929-53a4-4ed3-b43e-d487cf4e1833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:25.517413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b4929-53a4-4ed3-b43e-d487cf4e1833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:25.519029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-184b4929-53a4-4ed3-b43e-d487cf4e1833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:25.575186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:25.631878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b4929-53a4-4ed3-b43e-d487cf4e1833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:25.633368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b4929-53a4-4ed3-b43e-d487cf4e1833 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:25.652794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 656/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:27.991540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed355f6-309a-48b1-bc53-b5bc4aead17d None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:27.992065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed355f6-309a-48b1-bc53-b5bc4aead17d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:27.992065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed355f6-309a-48b1-bc53-b5bc4aead17d None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:27.994992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 656/1312] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:27.999229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5baa724-23e8-4818-84c0-80f69c8bf98a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:27.999667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5baa724-23e8-4818-84c0-80f69c8bf98a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:28.000090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5baa724-23e8-4818-84c0-80f69c8bf98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:28.089418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c5baa724-23e8-4818-84c0-80f69c8bf98a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:28.090859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c5baa724-23e8-4818-84c0-80f69c8bf98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:28.205393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 657/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:28.244804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed355f6-309a-48b1-bc53-b5bc4aead17d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:28.246301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-fed355f6-309a-48b1-bc53-b5bc4aead17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:28.358651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed355f6-309a-48b1-bc53-b5bc4aead17d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:28.445064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:28.448023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143156e-e751-4d00-b44a-ed76d981ca57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:28.448999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143156e-e751-4d00-b44a-ed76d981ca57 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:28.449649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143156e-e751-4d00-b44a-ed76d981ca57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:28.450330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143156e-e751-4d00-b44a-ed76d981ca57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:28.452430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5143156e-e751-4d00-b44a-ed76d981ca57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:28.579253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143156e-e751-4d00-b44a-ed76d981ca57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:28.580752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143156e-e751-4d00-b44a-ed76d981ca57 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:28.598277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 657/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:30.916053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1da6a-1512-4a02-9cff-5ccee3766085 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:30.916609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1da6a-1512-4a02-9cff-5ccee3766085 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:30.916609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1da6a-1512-4a02-9cff-5ccee3766085 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:30.917937 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 658/1315] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:30.923166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db2204-f633-4326-89f0-caa70d39ad8a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:30.923573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db2204-f633-4326-89f0-caa70d39ad8a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:30.923718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db2204-f633-4326-89f0-caa70d39ad8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:30.987822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-27db2204-f633-4326-89f0-caa70d39ad8a None None] MFA Rules not processed for user `acbec83fb2f04975b8e667d4223dfe33`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:30.989016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-27db2204-f633-4326-89f0-caa70d39ad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:31.039481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:31.082489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 658/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:31.122624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1da6a-1512-4a02-9cff-5ccee3766085 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:31.124174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1cc1da6a-1512-4a02-9cff-5ccee3766085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:31.133605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:31.225044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1da6a-1512-4a02-9cff-5ccee3766085 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:31.294631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e6eb1-2776-464f-9bde-4d560423d9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:31.295246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e6eb1-2776-464f-9bde-4d560423d9fd service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:31.295521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e6eb1-2776-464f-9bde-4d560423d9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:31.295787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e6eb1-2776-464f-9bde-4d560423d9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:31.297364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4a0e6eb1-2776-464f-9bde-4d560423d9fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:31.356597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:31.403227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e6eb1-2776-464f-9bde-4d560423d9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:31.404650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e6eb1-2776-464f-9bde-4d560423d9fd service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:31.422783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 659/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:33.967656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe338f-06ee-4748-bed4-b4c5c7b623c4 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:33.968189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe338f-06ee-4748-bed4-b4c5c7b623c4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:33.968189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe338f-06ee-4748-bed4-b4c5c7b623c4 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:33.970113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 659/1318] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:33.975348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42e5e7-8a97-4097-9c26-bd93f828e0bc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:33.975760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42e5e7-8a97-4097-9c26-bd93f828e0bc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:33.975898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42e5e7-8a97-4097-9c26-bd93f828e0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:34.039467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5b42e5e7-8a97-4097-9c26-bd93f828e0bc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:34.040671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5b42e5e7-8a97-4097-9c26-bd93f828e0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:34.133538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 660/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:34.175831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe338f-06ee-4748-bed4-b4c5c7b623c4 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:34.178030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-90fe338f-06ee-4748-bed4-b4c5c7b623c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:34.279803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe338f-06ee-4748-bed4-b4c5c7b623c4 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:34.350169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cfc7e-ee0b-4874-94d8-4d691bc751af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:34.350840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cfc7e-ee0b-4874-94d8-4d691bc751af service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:34.351149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cfc7e-ee0b-4874-94d8-4d691bc751af service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:34.351417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cfc7e-ee0b-4874-94d8-4d691bc751af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:34.352967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e91cfc7e-ee0b-4874-94d8-4d691bc751af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:34.359549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:34.458294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cfc7e-ee0b-4874-94d8-4d691bc751af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:34.459730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cfc7e-ee0b-4874-94d8-4d691bc751af service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:34.476747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 660/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:37.159355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34916be5-6157-4132-a479-2d798396e3c6 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:37.159980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34916be5-6157-4132-a479-2d798396e3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:37.159980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34916be5-6157-4132-a479-2d798396e3c6 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:37.161249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 661/1321] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:37.166472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ac4b8-5a0d-4dfe-9e62-8ee859b2f93a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:37.166902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ac4b8-5a0d-4dfe-9e62-8ee859b2f93a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:37.167067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ac4b8-5a0d-4dfe-9e62-8ee859b2f93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:37.232549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-9c5ac4b8-5a0d-4dfe-9e62-8ee859b2f93a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:37.233773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9c5ac4b8-5a0d-4dfe-9e62-8ee859b2f93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:37.295996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:37.336932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 661/1322] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:37.389137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ac0f3-ee5b-45b6-97d4-30980163e83c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:37.389445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ac0f3-ee5b-45b6-97d4-30980163e83c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:37.389716 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ac0f3-ee5b-45b6-97d4-30980163e83c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:37.391150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 662/1323] 10.222.0.20 () {54 vars in 965 bytes} [Wed Aug 10 09:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:37.397081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4300f-d002-47aa-bace-d65ca50a752d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:37.397371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4300f-d002-47aa-bace-d65ca50a752d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:37.397646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4300f-d002-47aa-bace-d65ca50a752d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:37.476731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f4e4300f-d002-47aa-bace-d65ca50a752d None None] MFA Rules not processed for user `ab9b7576a3834862af6abb7e0416f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:37.477971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f4e4300f-d002-47aa-bace-d65ca50a752d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:37.553554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 662/1324] 10.222.0.20 () {58 vars in 1062 bytes} [Wed Aug 10 09:55:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:37.562602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ac0f3-ee5b-45b6-97d4-30980163e83c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:37.564194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-030ac0f3-ee5b-45b6-97d4-30980163e83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:37.583943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:37.667794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ac0f3-ee5b-45b6-97d4-30980163e83c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:37.739037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c01642-17d6-409f-bc51-162879fae4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:37.739644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c01642-17d6-409f-bc51-162879fae4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:37.739928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c01642-17d6-409f-bc51-162879fae4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:37.740214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c01642-17d6-409f-bc51-162879fae4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:37.741793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-74c01642-17d6-409f-bc51-162879fae4f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:37.811514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:37.849331 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c01642-17d6-409f-bc51-162879fae4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:37.850754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c01642-17d6-409f-bc51-162879fae4f8 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:37.869128 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 663/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:40.139785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc888a0-068c-4df5-879a-41850d684002 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:40.140790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc888a0-068c-4df5-879a-41850d684002 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:40.140870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc888a0-068c-4df5-879a-41850d684002 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:40.142342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 663/1326] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:40.147681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46197-bf71-4c3f-b64c-2d3fa044324b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:40.148116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46197-bf71-4c3f-b64c-2d3fa044324b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:40.148248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46197-bf71-4c3f-b64c-2d3fa044324b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:40.226814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-afc46197-bf71-4c3f-b64c-2d3fa044324b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:40.228567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-afc46197-bf71-4c3f-b64c-2d3fa044324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:40.341505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 664/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:40.377592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc888a0-068c-4df5-879a-41850d684002 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:40.379470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0cc888a0-068c-4df5-879a-41850d684002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:40.441820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:40.485399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc888a0-068c-4df5-879a-41850d684002 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:40.571031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd8e23-817f-489e-b8c6-90cb347ce87a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:40.571542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd8e23-817f-489e-b8c6-90cb347ce87a service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:40.571818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd8e23-817f-489e-b8c6-90cb347ce87a service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:40.572089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd8e23-817f-489e-b8c6-90cb347ce87a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:40.573707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-31dd8e23-817f-489e-b8c6-90cb347ce87a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:40.681343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:40.684005 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd8e23-817f-489e-b8c6-90cb347ce87a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:40.685515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd8e23-817f-489e-b8c6-90cb347ce87a service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:40.704057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 664/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:42.769479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34782f39-fedb-4ece-9de6-c5c8d45f1745 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:42.769969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34782f39-fedb-4ece-9de6-c5c8d45f1745 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:42.770079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34782f39-fedb-4ece-9de6-c5c8d45f1745 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:42.771518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 665/1329] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:42.776637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713a81e-7473-42c0-866d-09525ff9fe6a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:42.777065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713a81e-7473-42c0-866d-09525ff9fe6a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:42.777185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713a81e-7473-42c0-866d-09525ff9fe6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:42.851607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-5713a81e-7473-42c0-866d-09525ff9fe6a None None] MFA Rules not processed for user `acbec83fb2f04975b8e667d4223dfe33`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:42.853200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5713a81e-7473-42c0-866d-09525ff9fe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:42.960546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 665/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:42.998127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34782f39-fedb-4ece-9de6-c5c8d45f1745 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:42.999703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-34782f39-fedb-4ece-9de6-c5c8d45f1745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:43.030120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:43.103432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34782f39-fedb-4ece-9de6-c5c8d45f1745 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:43.175702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd317b0f-8a50-4cff-875a-9155fdfe2d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:43.176304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd317b0f-8a50-4cff-875a-9155fdfe2d7c service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:43.176583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd317b0f-8a50-4cff-875a-9155fdfe2d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:43.176850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd317b0f-8a50-4cff-875a-9155fdfe2d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:43.178416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fd317b0f-8a50-4cff-875a-9155fdfe2d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:43.255802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:43.287765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd317b0f-8a50-4cff-875a-9155fdfe2d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:43.289186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd317b0f-8a50-4cff-875a-9155fdfe2d7c service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:43.307146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 666/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:45.721434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa1d97-2aff-4771-84b5-6c9642e31942 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:45.721947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa1d97-2aff-4771-84b5-6c9642e31942 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:45.721947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa1d97-2aff-4771-84b5-6c9642e31942 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:45.723271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 666/1332] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:45.728292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69f090-c12b-4c6f-910e-ca24e427b514 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:45.728676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69f090-c12b-4c6f-910e-ca24e427b514 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:45.728845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69f090-c12b-4c6f-910e-ca24e427b514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:45.792237 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3e69f090-c12b-4c6f-910e-ca24e427b514 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:45.793425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3e69f090-c12b-4c6f-910e-ca24e427b514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:45.886671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 667/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:45.922778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa1d97-2aff-4771-84b5-6c9642e31942 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:45.924314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-63fa1d97-2aff-4771-84b5-6c9642e31942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:45.991392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:46.024579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa1d97-2aff-4771-84b5-6c9642e31942 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:46.095577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5ff38-d7f0-49f0-b080-d5ec9435722f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:46.096166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5ff38-d7f0-49f0-b080-d5ec9435722f service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:46.096442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5ff38-d7f0-49f0-b080-d5ec9435722f service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:46.096700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5ff38-d7f0-49f0-b080-d5ec9435722f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:46.098261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3dd5ff38-d7f0-49f0-b080-d5ec9435722f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:46.228778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5ff38-d7f0-49f0-b080-d5ec9435722f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:46.230202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5ff38-d7f0-49f0-b080-d5ec9435722f service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:46.239934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:46.248512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 667/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:48.283747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bf769-f8c8-4822-8ee2-490c3708c166 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:48.284309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bf769-f8c8-4822-8ee2-490c3708c166 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:48.284309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bf769-f8c8-4822-8ee2-490c3708c166 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:48.285648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 668/1335] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:48.290779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15979fb-d236-4c81-be80-6f8ce6163e04 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:48.291202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15979fb-d236-4c81-be80-6f8ce6163e04 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:48.291352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15979fb-d236-4c81-be80-6f8ce6163e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:48.354647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d15979fb-d236-4c81-be80-6f8ce6163e04 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:48.355850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d15979fb-d236-4c81-be80-6f8ce6163e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:48.449499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 668/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:48.489675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bf769-f8c8-4822-8ee2-490c3708c166 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:48.491236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d95bf769-f8c8-4822-8ee2-490c3708c166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:48.531815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:48.594564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bf769-f8c8-4822-8ee2-490c3708c166 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:48.665918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0176-fe81-49eb-b83d-2fc4e181e377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:48.666514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0176-fe81-49eb-b83d-2fc4e181e377 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:48.666791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0176-fe81-49eb-b83d-2fc4e181e377 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:48.667088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0176-fe81-49eb-b83d-2fc4e181e377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:48.668670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-040a0176-fe81-49eb-b83d-2fc4e181e377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:48.749638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:48.774571 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0176-fe81-49eb-b83d-2fc4e181e377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:48.776091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0176-fe81-49eb-b83d-2fc4e181e377 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:48.794526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 669/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:51.244544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcade50-9871-4a29-bc4f-34d7a7aa3f51 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:51.245102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcade50-9871-4a29-bc4f-34d7a7aa3f51 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:51.245102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcade50-9871-4a29-bc4f-34d7a7aa3f51 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:51.246421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 669/1338] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:51.251611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61220d-f07c-40b7-8d45-c8a16f5781cf None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:51.252032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61220d-f07c-40b7-8d45-c8a16f5781cf None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:51.252169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61220d-f07c-40b7-8d45-c8a16f5781cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:51.316088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8e61220d-f07c-40b7-8d45-c8a16f5781cf None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:51.317263 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8e61220d-f07c-40b7-8d45-c8a16f5781cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:51.410303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 670/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:51.445604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcade50-9871-4a29-bc4f-34d7a7aa3f51 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:51.447146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1dcade50-9871-4a29-bc4f-34d7a7aa3f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:51.520946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:51.547933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcade50-9871-4a29-bc4f-34d7a7aa3f51 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:51.618537 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681938d-1a9f-4f34-a218-1f549f9638f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:51.619441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681938d-1a9f-4f34-a218-1f549f9638f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:51.619719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681938d-1a9f-4f34-a218-1f549f9638f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:51.619991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681938d-1a9f-4f34-a218-1f549f9638f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:51.621580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b681938d-1a9f-4f34-a218-1f549f9638f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:51.729084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681938d-1a9f-4f34-a218-1f549f9638f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:51.730498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681938d-1a9f-4f34-a218-1f549f9638f6 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:51.748021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 670/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:55.720549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80c1cc-9399-49bb-8ba9-8a32a7fc19da None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:55.721100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80c1cc-9399-49bb-8ba9-8a32a7fc19da None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:55.721100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80c1cc-9399-49bb-8ba9-8a32a7fc19da None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:55.722514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 671/1341] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:55.727797 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117c7a2-dabe-46fb-8d84-b3ca24592145 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:55.728208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117c7a2-dabe-46fb-8d84-b3ca24592145 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:55.728421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117c7a2-dabe-46fb-8d84-b3ca24592145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:55.734316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:55.795232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d117c7a2-dabe-46fb-8d84-b3ca24592145 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:55.796952 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d117c7a2-dabe-46fb-8d84-b3ca24592145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:55.894074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 671/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:55.932397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80c1cc-9399-49bb-8ba9-8a32a7fc19da None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:55.934034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bc80c1cc-9399-49bb-8ba9-8a32a7fc19da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:55.982251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:56.038249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80c1cc-9399-49bb-8ba9-8a32a7fc19da None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:56.110946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68875da6-0429-46f6-bebf-7da8e03ffa66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:56.111575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68875da6-0429-46f6-bebf-7da8e03ffa66 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:56.111847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68875da6-0429-46f6-bebf-7da8e03ffa66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:56.112109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68875da6-0429-46f6-bebf-7da8e03ffa66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:56.113687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-68875da6-0429-46f6-bebf-7da8e03ffa66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:56.207616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:56.224768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68875da6-0429-46f6-bebf-7da8e03ffa66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:56.226197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68875da6-0429-46f6-bebf-7da8e03ffa66 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:56.245586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 672/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:55:58.842182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbbeba-ff90-4cd8-a938-b45d4ee32517 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:58.842746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbbeba-ff90-4cd8-a938-b45d4ee32517 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:58.842746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbbeba-ff90-4cd8-a938-b45d4ee32517 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:58.844244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 672/1344] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:55:58.849258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab825d4b-5c6d-4f42-a1f3-7b123916d733 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:58.849645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab825d4b-5c6d-4f42-a1f3-7b123916d733 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:58.849803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab825d4b-5c6d-4f42-a1f3-7b123916d733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:58.922917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ab825d4b-5c6d-4f42-a1f3-7b123916d733 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:55:58.924154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ab825d4b-5c6d-4f42-a1f3-7b123916d733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:59.028140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 673/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:55:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:55:59.065979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbbeba-ff90-4cd8-a938-b45d4ee32517 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:55:59.067665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-61bbbeba-ff90-4cd8-a938-b45d4ee32517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:59.152014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:55:59.170476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbbeba-ff90-4cd8-a938-b45d4ee32517 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:55:59.246409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5464c43-b896-40c5-916a-fab4e0fa5a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:55:59.247030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5464c43-b896-40c5-916a-fab4e0fa5a0e service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:55:59.247302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5464c43-b896-40c5-916a-fab4e0fa5a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:55:59.247563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5464c43-b896-40c5-916a-fab4e0fa5a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:55:59.249158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b5464c43-b896-40c5-916a-fab4e0fa5a0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:55:59.354607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5464c43-b896-40c5-916a-fab4e0fa5a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:55:59.356058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5464c43-b896-40c5-916a-fab4e0fa5a0e service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:55:59.373665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 673/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:55:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:03.293375 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be4436-71aa-4cb3-a0ab-7ede94a16dff None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:03.293900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be4436-71aa-4cb3-a0ab-7ede94a16dff None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:03.293900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be4436-71aa-4cb3-a0ab-7ede94a16dff None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:03.295299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 674/1347] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:03.301197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0b801-45fb-4049-aa7c-7859aed20b65 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:03.301620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0b801-45fb-4049-aa7c-7859aed20b65 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:03.301758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0b801-45fb-4049-aa7c-7859aed20b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:03.317671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:03.371376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-adb0b801-45fb-4049-aa7c-7859aed20b65 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:03.372587 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-adb0b801-45fb-4049-aa7c-7859aed20b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:03.474591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 674/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:03.512902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be4436-71aa-4cb3-a0ab-7ede94a16dff None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:03.514452 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-81be4436-71aa-4cb3-a0ab-7ede94a16dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:03.572277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:03.619771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be4436-71aa-4cb3-a0ab-7ede94a16dff None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:03.712437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac23885-e7a9-4f8d-ad4f-3509e5bc1241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:03.713242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac23885-e7a9-4f8d-ad4f-3509e5bc1241 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:03.713686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac23885-e7a9-4f8d-ad4f-3509e5bc1241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:03.714006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac23885-e7a9-4f8d-ad4f-3509e5bc1241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:03.716054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2ac23885-e7a9-4f8d-ad4f-3509e5bc1241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:03.831087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:03.835005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac23885-e7a9-4f8d-ad4f-3509e5bc1241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:03.836975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac23885-e7a9-4f8d-ad4f-3509e5bc1241 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:03.856422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 675/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:08.693737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8e0fb-ade8-4c00-95e9-d554dd624e48 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:08.695597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8e0fb-ade8-4c00-95e9-d554dd624e48 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:08.695597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8e0fb-ade8-4c00-95e9-d554dd624e48 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:08.697470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 675/1350] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 09:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:08.716613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ee5da-f66d-49bf-b1c0-88fd56580fdf None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:08.716613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ee5da-f66d-49bf-b1c0-88fd56580fdf None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:08.716613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ee5da-f66d-49bf-b1c0-88fd56580fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:08.820402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1e5ee5da-f66d-49bf-b1c0-88fd56580fdf None None] MFA Rules not processed for user `d3d7b3cb498a4da18e203456e09b59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:08.821588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1e5ee5da-f66d-49bf-b1c0-88fd56580fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:08.966309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 676/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 09:56:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:12.919330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce8252-2da3-4223-a392-96d8654fd518 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:12.919916 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce8252-2da3-4223-a392-96d8654fd518 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:12.919916 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce8252-2da3-4223-a392-96d8654fd518 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:12.921874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 676/1352] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 09:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:12.927135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442696d4-d08c-4152-8c09-325bc666cdf9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:12.927699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442696d4-d08c-4152-8c09-325bc666cdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:12.928016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442696d4-d08c-4152-8c09-325bc666cdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:12.984793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:13.009930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-442696d4-d08c-4152-8c09-325bc666cdf9 None None] MFA Rules not processed for user `d3d7b3cb498a4da18e203456e09b59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:13.011180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-442696d4-d08c-4152-8c09-325bc666cdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:13.125724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 677/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 09:56:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:56:13.472990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=59 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:14.520656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce54c-0f56-4bca-9bcd-84ffb8edff4a None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:14.521209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce54c-0f56-4bca-9bcd-84ffb8edff4a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:14.521544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce54c-0f56-4bca-9bcd-84ffb8edff4a None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:14.522963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 677/1354] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 09:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:14.528849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370e2e1-a864-4304-9a3f-31b49c489a48 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:14.529381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370e2e1-a864-4304-9a3f-31b49c489a48 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:14.529381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370e2e1-a864-4304-9a3f-31b49c489a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:14.605774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5370e2e1-a864-4304-9a3f-31b49c489a48 None None] MFA Rules not processed for user `d3d7b3cb498a4da18e203456e09b59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:14.606980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5370e2e1-a864-4304-9a3f-31b49c489a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:14.648974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:14.715384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 678/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 09:56:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:18.290338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5e2a-1f99-4ca4-be9c-6aac00f29373 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:18.291226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5e2a-1f99-4ca4-be9c-6aac00f29373 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:18.291667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5e2a-1f99-4ca4-be9c-6aac00f29373 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:18.293951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 678/1356] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:18.299413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f79f50-48a6-44eb-900d-c5726e5c402b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:18.300549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f79f50-48a6-44eb-900d-c5726e5c402b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:18.301049 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f79f50-48a6-44eb-900d-c5726e5c402b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:18.375675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-49f79f50-48a6-44eb-900d-c5726e5c402b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:18.377015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-49f79f50-48a6-44eb-900d-c5726e5c402b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:18.485489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 679/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:18.547755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bd447-5190-48b4-a5ae-9a2b65690f67 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:18.548179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bd447-5190-48b4-a5ae-9a2b65690f67 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:18.548577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bd447-5190-48b4-a5ae-9a2b65690f67 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:18.550022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 679/1358] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 10 09:56:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:18.560361 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94647966-e605-453d-b0f0-04cd9076bd1e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:18.560904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94647966-e605-453d-b0f0-04cd9076bd1e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:18.561285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94647966-e605-453d-b0f0-04cd9076bd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:18.579023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:18.643052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-94647966-e605-453d-b0f0-04cd9076bd1e None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:18.644406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-94647966-e605-453d-b0f0-04cd9076bd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:18.773152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 680/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 10 09:56:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:56:18.782614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bd447-5190-48b4-a5ae-9a2b65690f67 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:18.784137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5d3bd447-5190-48b4-a5ae-9a2b65690f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:18.936096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bd447-5190-48b4-a5ae-9a2b65690f67 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:19.013495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:19.057364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031044af-069b-4462-8ed1-93fa3551d2ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:19.057975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031044af-069b-4462-8ed1-93fa3551d2ae service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:19.058386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031044af-069b-4462-8ed1-93fa3551d2ae service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:19.058659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031044af-069b-4462-8ed1-93fa3551d2ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:19.060419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-031044af-069b-4462-8ed1-93fa3551d2ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:19.158429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031044af-069b-4462-8ed1-93fa3551d2ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:19.160626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031044af-069b-4462-8ed1-93fa3551d2ae service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:19.179696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 680/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:21.387671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5fe2d-3c61-4253-b9f6-42d108ab8f5a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:21.388219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5fe2d-3c61-4253-b9f6-42d108ab8f5a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:21.388219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5fe2d-3c61-4253-b9f6-42d108ab8f5a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:21.389474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 681/1361] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:21.395246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be184a30-1c39-42da-87fc-758092ff34bd None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:21.396094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be184a30-1c39-42da-87fc-758092ff34bd None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:21.396574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be184a30-1c39-42da-87fc-758092ff34bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:21.466315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-be184a30-1c39-42da-87fc-758092ff34bd None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:21.467671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-be184a30-1c39-42da-87fc-758092ff34bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:21.506742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:21.578985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 681/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:21.624049 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5fe2d-3c61-4253-b9f6-42d108ab8f5a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:21.626770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2fb5fe2d-3c61-4253-b9f6-42d108ab8f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:21.695989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:21.798449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5fe2d-3c61-4253-b9f6-42d108ab8f5a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:21.941057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f452b-baa9-4be6-978c-df73e3255c20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:21.941874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f452b-baa9-4be6-978c-df73e3255c20 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:21.942227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f452b-baa9-4be6-978c-df73e3255c20 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:21.942574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f452b-baa9-4be6-978c-df73e3255c20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:21.944275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-404f452b-baa9-4be6-978c-df73e3255c20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:21.967802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:22.037992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f452b-baa9-4be6-978c-df73e3255c20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:22.039634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f452b-baa9-4be6-978c-df73e3255c20 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:22.058501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 682/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:24.402885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd2f19-f581-4459-9594-9d00c187d076 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:24.403834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd2f19-f581-4459-9594-9d00c187d076 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:24.404116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd2f19-f581-4459-9594-9d00c187d076 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:24.405447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 682/1364] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:24.411550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884a871-c49e-47bf-b55a-16454b1d3263 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:24.412958 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884a871-c49e-47bf-b55a-16454b1d3263 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:24.413236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884a871-c49e-47bf-b55a-16454b1d3263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:24.484952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7884a871-c49e-47bf-b55a-16454b1d3263 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:24.486073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7884a871-c49e-47bf-b55a-16454b1d3263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:24.594378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:24.597059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 683/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:56:24.642423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e741e1-edce-4c47-870f-0f4d6093f50e None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:24.643129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e741e1-edce-4c47-870f-0f4d6093f50e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:24.643516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e741e1-edce-4c47-870f-0f4d6093f50e None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:24.645020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 683/1366] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 10 09:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:24.650570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2d088-eaba-4d48-ae5f-65f646206e5f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:24.650855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2d088-eaba-4d48-ae5f-65f646206e5f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:24.651161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2d088-eaba-4d48-ae5f-65f646206e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:24.730309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3ba2d088-eaba-4d48-ae5f-65f646206e5f None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:24.731535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3ba2d088-eaba-4d48-ae5f-65f646206e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:24.855977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 684/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 10 09:56:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:24.864059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e741e1-edce-4c47-870f-0f4d6093f50e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:24.865561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-21e741e1-edce-4c47-870f-0f4d6093f50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:25.023922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e741e1-edce-4c47-870f-0f4d6093f50e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:25.047302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:25.145831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f8105-c017-40a4-b534-a41776369ec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:25.146503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f8105-c017-40a4-b534-a41776369ec1 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:25.146885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f8105-c017-40a4-b534-a41776369ec1 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:25.147344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f8105-c017-40a4-b534-a41776369ec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:25.149064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ef8f8105-c017-40a4-b534-a41776369ec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:25.248554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f8105-c017-40a4-b534-a41776369ec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:25.250337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f8105-c017-40a4-b534-a41776369ec1 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:25.269582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 684/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:26.231235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed41163-94e3-477c-bb96-7bd9d09e51d6 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:26.231770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed41163-94e3-477c-bb96-7bd9d09e51d6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:26.231770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed41163-94e3-477c-bb96-7bd9d09e51d6 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:26.233105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 685/1369] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:26.238484 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20586a-3d69-4b2f-b8bb-c94c52028c41 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:26.239277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20586a-3d69-4b2f-b8bb-c94c52028c41 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:26.239619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20586a-3d69-4b2f-b8bb-c94c52028c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:26.274074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:26.310831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ef20586a-3d69-4b2f-b8bb-c94c52028c41 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:26.312668 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ef20586a-3d69-4b2f-b8bb-c94c52028c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:26.422985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 685/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:26.783963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f203f54-92f6-4d71-982a-53062f1551b5 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:26.784372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f203f54-92f6-4d71-982a-53062f1551b5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:26.784703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f203f54-92f6-4d71-982a-53062f1551b5 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:26.786347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 686/1371] 10.222.0.20 () {54 vars in 964 bytes} [Wed Aug 10 09:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:26.791658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023f177-6f84-418c-9035-0b3fd9535b59 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:26.791658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023f177-6f84-418c-9035-0b3fd9535b59 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:26.792040 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023f177-6f84-418c-9035-0b3fd9535b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:26.870971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6023f177-6f84-418c-9035-0b3fd9535b59 None None] MFA Rules not processed for user `f1e3ba8d0c7d4f8190cd0b5b057f5638`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:26.872284 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6023f177-6f84-418c-9035-0b3fd9535b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:26.927944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:26.962442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 686/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Aug 10 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:26.970479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f203f54-92f6-4d71-982a-53062f1551b5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:26.972082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6f203f54-92f6-4d71-982a-53062f1551b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:27.026613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:27.085675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f203f54-92f6-4d71-982a-53062f1551b5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:27.166330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c9685-5ff2-4478-b680-35ea78647282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:27.166932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c9685-5ff2-4478-b680-35ea78647282 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:27.167235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c9685-5ff2-4478-b680-35ea78647282 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:27.167502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c9685-5ff2-4478-b680-35ea78647282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:27.169116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b01c9685-5ff2-4478-b680-35ea78647282 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:27.275110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:27.289934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c9685-5ff2-4478-b680-35ea78647282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:27.297080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c9685-5ff2-4478-b680-35ea78647282 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:27.310328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 687/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:27.404929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c9fc-1cce-4c02-a6df-400d0556f21f None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:27.405354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c9fc-1cce-4c02-a6df-400d0556f21f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:27.405693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c9fc-1cce-4c02-a6df-400d0556f21f None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:27.411687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 687/1374] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:27.415200 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b641e9-67b9-40ff-9921-40e5995151e0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:27.415494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b641e9-67b9-40ff-9921-40e5995151e0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:27.415772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b641e9-67b9-40ff-9921-40e5995151e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:27.486457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-52b641e9-67b9-40ff-9921-40e5995151e0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:27.488026 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-52b641e9-67b9-40ff-9921-40e5995151e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:27.596624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 688/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:27.641717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31c9fc-1cce-4c02-a6df-400d0556f21f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:27.646792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9b31c9fc-1cce-4c02-a6df-400d0556f21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:27.801198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31c9fc-1cce-4c02-a6df-400d0556f21f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:27.851516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:27.914382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb5660-13c9-42b4-afc0-38f56e4b85d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:27.915011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb5660-13c9-42b4-afc0-38f56e4b85d0 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:27.915289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb5660-13c9-42b4-afc0-38f56e4b85d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:27.915557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb5660-13c9-42b4-afc0-38f56e4b85d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:27.917154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bfdb5660-13c9-42b4-afc0-38f56e4b85d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:28.008470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb5660-13c9-42b4-afc0-38f56e4b85d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:28.010075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb5660-13c9-42b4-afc0-38f56e4b85d0 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:28.028099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 688/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:30.114090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c689f6f-4e91-491a-be3d-24d8bdf3b94f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:30.114556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c689f6f-4e91-491a-be3d-24d8bdf3b94f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:30.114660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c689f6f-4e91-491a-be3d-24d8bdf3b94f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:30.116035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 689/1377] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:30.121406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaee806-d1b2-4483-8d86-9af2df667693 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:30.121831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaee806-d1b2-4483-8d86-9af2df667693 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:30.121964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaee806-d1b2-4483-8d86-9af2df667693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:30.192119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-cbaee806-d1b2-4483-8d86-9af2df667693 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:30.193326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-cbaee806-d1b2-4483-8d86-9af2df667693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:30.199908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:30.296090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 689/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:30.341736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c689f6f-4e91-491a-be3d-24d8bdf3b94f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:30.343881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4c689f6f-4e91-491a-be3d-24d8bdf3b94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:30.417180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:30.502167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c689f6f-4e91-491a-be3d-24d8bdf3b94f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:30.619394 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039ee70-1ff1-4ee1-a0f6-5675ecb27c11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:30.620250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039ee70-1ff1-4ee1-a0f6-5675ecb27c11 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:30.620525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039ee70-1ff1-4ee1-a0f6-5675ecb27c11 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:30.620793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039ee70-1ff1-4ee1-a0f6-5675ecb27c11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:30.622377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e039ee70-1ff1-4ee1-a0f6-5675ecb27c11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:30.641199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:30.727138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039ee70-1ff1-4ee1-a0f6-5675ecb27c11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:30.728807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039ee70-1ff1-4ee1-a0f6-5675ecb27c11 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:30.747088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 690/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:32.799820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5cc1a-1994-469f-84e4-17b1758710c4 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:32.800422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5cc1a-1994-469f-84e4-17b1758710c4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:32.800422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5cc1a-1994-469f-84e4-17b1758710c4 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:32.801846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 690/1380] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:32.806822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207de049-e73a-4907-b953-24029305852d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:32.807303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207de049-e73a-4907-b953-24029305852d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:32.807394 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207de049-e73a-4907-b953-24029305852d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:32.999598 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-207de049-e73a-4907-b953-24029305852d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:33.000894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-207de049-e73a-4907-b953-24029305852d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:33.094787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:33.103312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 691/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:33.146688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5cc1a-1994-469f-84e4-17b1758710c4 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:33.148396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-caf5cc1a-1994-469f-84e4-17b1758710c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:33.298638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:33.309726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5cc1a-1994-469f-84e4-17b1758710c4 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:33.433348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677deb4b-f215-452d-b5f7-fbf1f9c1b9d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:33.434084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677deb4b-f215-452d-b5f7-fbf1f9c1b9d0 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:33.434459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677deb4b-f215-452d-b5f7-fbf1f9c1b9d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:33.434843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677deb4b-f215-452d-b5f7-fbf1f9c1b9d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:33.436506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-677deb4b-f215-452d-b5f7-fbf1f9c1b9d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:33.535722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677deb4b-f215-452d-b5f7-fbf1f9c1b9d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:33.537459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677deb4b-f215-452d-b5f7-fbf1f9c1b9d0 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:33.548433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:33.557869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 691/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 09:56:35.638392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d46c9-a059-410d-922a-60810cbf436b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:35.638944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d46c9-a059-410d-922a-60810cbf436b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:35.639200 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d46c9-a059-410d-922a-60810cbf436b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:35.641336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 692/1383] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:35.647703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285e17-94eb-4296-a8c4-ddf1baf16624 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:35.648096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285e17-94eb-4296-a8c4-ddf1baf16624 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:35.648283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285e17-94eb-4296-a8c4-ddf1baf16624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:35.716070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6a285e17-94eb-4296-a8c4-ddf1baf16624 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:35.717208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6a285e17-94eb-4296-a8c4-ddf1baf16624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:35.822510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 692/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:35.864464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d46c9-a059-410d-922a-60810cbf436b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:35.866374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d85d46c9-a059-410d-922a-60810cbf436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:36.024636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d46c9-a059-410d-922a-60810cbf436b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:36.091515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:36.150004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27cb2b-c67f-4b5f-a68d-edf6f3b5c514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:36.150761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27cb2b-c67f-4b5f-a68d-edf6f3b5c514 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:36.151142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27cb2b-c67f-4b5f-a68d-edf6f3b5c514 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:36.151486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27cb2b-c67f-4b5f-a68d-edf6f3b5c514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:36.153539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3a27cb2b-c67f-4b5f-a68d-edf6f3b5c514 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:36.264637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27cb2b-c67f-4b5f-a68d-edf6f3b5c514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:36.266682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27cb2b-c67f-4b5f-a68d-edf6f3b5c514 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:36.289259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 693/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:38.282734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dc02a-97c0-4814-a8a6-9e86e6cb40e2 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:38.283376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dc02a-97c0-4814-a8a6-9e86e6cb40e2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:38.283376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dc02a-97c0-4814-a8a6-9e86e6cb40e2 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:38.284548 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 693/1386] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:38.290821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f168e8-338f-4d29-b467-ed5492c024c0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:38.290821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f168e8-338f-4d29-b467-ed5492c024c0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:38.290821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f168e8-338f-4d29-b467-ed5492c024c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:38.356221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-69f168e8-338f-4d29-b467-ed5492c024c0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:38.357340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-69f168e8-338f-4d29-b467-ed5492c024c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:38.390927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:38.457272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 694/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:38.504370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dc02a-97c0-4814-a8a6-9e86e6cb40e2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:38.505909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-171dc02a-97c0-4814-a8a6-9e86e6cb40e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:38.592526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:38.672729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dc02a-97c0-4814-a8a6-9e86e6cb40e2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:38.794974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9599d-c847-4237-8f7d-259a9036750a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:38.795583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9599d-c847-4237-8f7d-259a9036750a service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:38.795992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9599d-c847-4237-8f7d-259a9036750a service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:38.796152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9599d-c847-4237-8f7d-259a9036750a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:38.797810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-fbd9599d-c847-4237-8f7d-259a9036750a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:38.836544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:38.888661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9599d-c847-4237-8f7d-259a9036750a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:38.890797 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9599d-c847-4237-8f7d-259a9036750a service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:38.908374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 694/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 09:56:40.932946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9f360-d9df-49f4-9a4a-48e8f626e058 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:40.933542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9f360-d9df-49f4-9a4a-48e8f626e058 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:40.933542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9f360-d9df-49f4-9a4a-48e8f626e058 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:40.935007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 695/1389] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:40.940624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6968d-f0a1-4ff2-899b-34e572a4667c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:40.941016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6968d-f0a1-4ff2-899b-34e572a4667c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:40.941178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6968d-f0a1-4ff2-899b-34e572a4667c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:41.007704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-dfb6968d-f0a1-4ff2-899b-34e572a4667c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:41.008897 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-dfb6968d-f0a1-4ff2-899b-34e572a4667c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:41.103883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 695/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:41.151882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9f360-d9df-49f4-9a4a-48e8f626e058 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:41.153403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e2e9f360-d9df-49f4-9a4a-48e8f626e058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:41.297245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9f360-d9df-49f4-9a4a-48e8f626e058 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:41.313665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:41.439791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef60e1-dc71-40ee-b15a-666be2856f5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:41.440401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef60e1-dc71-40ee-b15a-666be2856f5f service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:41.440677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef60e1-dc71-40ee-b15a-666be2856f5f service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:41.440949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef60e1-dc71-40ee-b15a-666be2856f5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:41.442539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-44ef60e1-dc71-40ee-b15a-666be2856f5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:41.535061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef60e1-dc71-40ee-b15a-666be2856f5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:41.536735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef60e1-dc71-40ee-b15a-666be2856f5f service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:41.554166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 696/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 09:56:42.398810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887c54-4f07-42c3-87a2-fdc1e1b4361d None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:42.398810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887c54-4f07-42c3-87a2-fdc1e1b4361d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:42.399567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887c54-4f07-42c3-87a2-fdc1e1b4361d None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:42.400581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 696/1392] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:42.405788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6ed50-62f1-4414-9438-6dd4d248e777 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:42.406248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6ed50-62f1-4414-9438-6dd4d248e777 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:42.406338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6ed50-62f1-4414-9438-6dd4d248e777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:42.431904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:42.472641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-72c6ed50-62f1-4414-9438-6dd4d248e777 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:42.473831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-72c6ed50-62f1-4414-9438-6dd4d248e777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:42.571399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 697/1393] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:42.626680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde25065-36fe-4d6b-aa73-50adf51f1e3d None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:42.627085 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde25065-36fe-4d6b-aa73-50adf51f1e3d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:42.627364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde25065-36fe-4d6b-aa73-50adf51f1e3d None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:42.628686 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 697/1394] 10.222.0.20 () {54 vars in 967 bytes} [Wed Aug 10 09:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:42.633730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2f003-f72e-4caf-8140-fd8afe6f61e9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:42.634028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2f003-f72e-4caf-8140-fd8afe6f61e9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:42.634280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2f003-f72e-4caf-8140-fd8afe6f61e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:42.714877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dec2f003-f72e-4caf-8140-fd8afe6f61e9 None None] MFA Rules not processed for user `b8f456f0d15b4d068cfcd6476563247f`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:42.716038 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dec2f003-f72e-4caf-8140-fd8afe6f61e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:42.756901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:42.794161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 698/1395] 10.222.0.20 () {58 vars in 1064 bytes} [Wed Aug 10 09:56:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:42.802072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde25065-36fe-4d6b-aa73-50adf51f1e3d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:42.803661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-cde25065-36fe-4d6b-aa73-50adf51f1e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:42.817709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:42.913909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde25065-36fe-4d6b-aa73-50adf51f1e3d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:42.985427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80773746-d664-44cc-8e65-f09a23abaf55 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8f456f0d15b4d068cfcd6476563247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:42.985956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80773746-d664-44cc-8e65-f09a23abaf55 service glance] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:42.986233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80773746-d664-44cc-8e65-f09a23abaf55 service glance] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:42.986497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80773746-d664-44cc-8e65-f09a23abaf55 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:42.988103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-80773746-d664-44cc-8e65-f09a23abaf55 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:43.054744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:43.096745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80773746-d664-44cc-8e65-f09a23abaf55 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:43.098172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80773746-d664-44cc-8e65-f09a23abaf55 service glance] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:43.115824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 698/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:43.625183 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c84a5-b2da-436c-a3ca-53b7b0839ffe None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:43.625576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c84a5-b2da-436c-a3ca-53b7b0839ffe None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:43.625736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c84a5-b2da-436c-a3ca-53b7b0839ffe None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:43.627122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 699/1397] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:43.632201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4c0ae-2371-4932-bfb5-e46690f0a1cc None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:43.632656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4c0ae-2371-4932-bfb5-e46690f0a1cc None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:43.632757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4c0ae-2371-4932-bfb5-e46690f0a1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:43.698475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-67c4c0ae-2371-4932-bfb5-e46690f0a1cc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:43.699698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-67c4c0ae-2371-4932-bfb5-e46690f0a1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:43.815994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 699/1398] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:43.862868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c84a5-b2da-436c-a3ca-53b7b0839ffe None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:43.864707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-461c84a5-b2da-436c-a3ca-53b7b0839ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:43.869870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d075fc-7581-47e0-b10e-cd54e89e4ff4 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:43.870220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d075fc-7581-47e0-b10e-cd54e89e4ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:43.870805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d075fc-7581-47e0-b10e-cd54e89e4ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:43.897749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:43.940805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-24d075fc-7581-47e0-b10e-cd54e89e4ff4 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:43.942288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-24d075fc-7581-47e0-b10e-cd54e89e4ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:44.014558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c84a5-b2da-436c-a3ca-53b7b0839ffe None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:44.028520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 700/1399] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:56:44.063677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:44.077859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997471b1-dc4f-4ab7-9c30-49c2e9c7bb17 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:44.078161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997471b1-dc4f-4ab7-9c30-49c2e9c7bb17 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:44.078431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997471b1-dc4f-4ab7-9c30-49c2e9c7bb17 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:44.079832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 701/1400] 10.222.0.20 () {54 vars in 943 bytes} [Wed Aug 10 09:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:56:44.086038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5afa-999f-4eb0-a874-697e2e404486 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:44.086318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5afa-999f-4eb0-a874-697e2e404486 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:44.086590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5afa-999f-4eb0-a874-697e2e404486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:44.131462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:44.132067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:44.132346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:44.132617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:44.134236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:44.164660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c63e5afa-999f-4eb0-a874-697e2e404486 None None] MFA Rules not processed for user `af29f3451f654f53ac98239a2b28d128`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:44.165865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c63e5afa-999f-4eb0-a874-697e2e404486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:44.217612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:44.224351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:44.225950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:44.244769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 700/1401] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:44.274203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 702/1402] 10.222.0.20 () {58 vars in 1040 bytes} [Wed Aug 10 09:56:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:56:44.285215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:44.286417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:44.345180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:44.381542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fc0a8-188c-4e47-b2ab-09b74ae24154 service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:44.492900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d968c08-c98d-4403-becb-206108ecf9c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:44.493502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968c08-c98d-4403-becb-206108ecf9c2 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:44.493780 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968c08-c98d-4403-becb-206108ecf9c2 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:44.494045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968c08-c98d-4403-becb-206108ecf9c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:44.495953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8d968c08-c98d-4403-becb-206108ecf9c2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:44.583915 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d968c08-c98d-4403-becb-206108ecf9c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:44.585540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d968c08-c98d-4403-becb-206108ecf9c2 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:44.590373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:44.594733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 701/1403] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 09:56:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 09:56:46.305871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd77a0-e017-48bb-8636-5a1b4ad1e800 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:46.305871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd77a0-e017-48bb-8636-5a1b4ad1e800 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:46.305871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd77a0-e017-48bb-8636-5a1b4ad1e800 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:46.305871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 703/1404] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:46.308681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d095bf-540e-49c1-8edb-83dcbb98abc6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:46.309152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d095bf-540e-49c1-8edb-83dcbb98abc6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:46.309235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d095bf-540e-49c1-8edb-83dcbb98abc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:46.397949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-82d095bf-540e-49c1-8edb-83dcbb98abc6 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:46.399384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-82d095bf-540e-49c1-8edb-83dcbb98abc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:46.507542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 702/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:46.552553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd77a0-e017-48bb-8636-5a1b4ad1e800 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:46.554167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-40fd77a0-e017-48bb-8636-5a1b4ad1e800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:46.717357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd77a0-e017-48bb-8636-5a1b4ad1e800 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:46.746167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:46.871131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040a3ae-64c1-4a1a-b458-f7b3d851200e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:46.871838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040a3ae-64c1-4a1a-b458-f7b3d851200e service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:46.872117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040a3ae-64c1-4a1a-b458-f7b3d851200e service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:46.874203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040a3ae-64c1-4a1a-b458-f7b3d851200e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:46.881810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3040a3ae-64c1-4a1a-b458-f7b3d851200e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:47.005035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040a3ae-64c1-4a1a-b458-f7b3d851200e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:47.006706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040a3ae-64c1-4a1a-b458-f7b3d851200e service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:47.035652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 704/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:49.096462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54914a-4a20-4f2a-bf53-2bb3a0bd74cd None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:49.096994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54914a-4a20-4f2a-bf53-2bb3a0bd74cd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:49.096994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54914a-4a20-4f2a-bf53-2bb3a0bd74cd None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:49.098332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 703/1407] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:56:49.103557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298140f-40ad-44ee-baaf-b8ce5dabaa18 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:49.103961 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298140f-40ad-44ee-baaf-b8ce5dabaa18 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:49.104134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298140f-40ad-44ee-baaf-b8ce5dabaa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:49.149506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:49.176168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-1298140f-40ad-44ee-baaf-b8ce5dabaa18 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:56:49.177361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1298140f-40ad-44ee-baaf-b8ce5dabaa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:49.288432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 705/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:56:49.332995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54914a-4a20-4f2a-bf53-2bb3a0bd74cd None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:56:49.334521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8f54914a-4a20-4f2a-bf53-2bb3a0bd74cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:49.431159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:49.506053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54914a-4a20-4f2a-bf53-2bb3a0bd74cd None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:56:49.629009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e416454-f087-412e-af77-8bf2c528b8ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:56:49.629632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e416454-f087-412e-af77-8bf2c528b8ea service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:56:49.629902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e416454-f087-412e-af77-8bf2c528b8ea service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:56:49.630245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e416454-f087-412e-af77-8bf2c528b8ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:56:49.631891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4e416454-f087-412e-af77-8bf2c528b8ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:56:49.679998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:49.732492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e416454-f087-412e-af77-8bf2c528b8ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:56:49.734485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e416454-f087-412e-af77-8bf2c528b8ea service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:56:49.755224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 704/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:56:59.286585 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=51 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:56:59.750984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=27 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:07.164331 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697ec33-eb77-4b25-856e-dd210860b4e6 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:07.164881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697ec33-eb77-4b25-856e-dd210860b4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:07.164881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697ec33-eb77-4b25-856e-dd210860b4e6 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:07.166128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 706/1410] 10.222.0.20 () {54 vars in 918 bytes} [Wed Aug 10 09:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 09:57:07.171373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a3139-f774-4d9c-bb13-435361cb1775 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:07.171820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a3139-f774-4d9c-bb13-435361cb1775 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:07.171913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a3139-f774-4d9c-bb13-435361cb1775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:07.246518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b25a3139-f774-4d9c-bb13-435361cb1775 None None] MFA Rules not processed for user `d3d7b3cb498a4da18e203456e09b59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:07.247725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b25a3139-f774-4d9c-bb13-435361cb1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:07.354507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 705/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 10 09:57:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:07.369250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36294-3f4c-451d-bb02-bb9a3c077d8d None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:07.369526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36294-3f4c-451d-bb02-bb9a3c077d8d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:07.369793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36294-3f4c-451d-bb02-bb9a3c077d8d None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:07.371380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 707/1412] 10.222.0.20 () {54 vars in 918 bytes} [Wed Aug 10 09:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:07.376366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05055c4-bd66-4977-a4f3-3713da7021ec None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:07.376647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05055c4-bd66-4977-a4f3-3713da7021ec None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:07.376916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05055c4-bd66-4977-a4f3-3713da7021ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:07.446720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:07.449688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a05055c4-bd66-4977-a4f3-3713da7021ec None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:07.451428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a05055c4-bd66-4977-a4f3-3713da7021ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:07.566773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 706/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 10 09:57:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:57:10.025432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544c21a-b719-4277-9854-eef3f6982a19 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:10.025973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544c21a-b719-4277-9854-eef3f6982a19 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:10.025973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544c21a-b719-4277-9854-eef3f6982a19 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:10.027336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:10.032647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090dce-9caa-48e5-a70e-11ec5ee5177f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:10.033070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090dce-9caa-48e5-a70e-11ec5ee5177f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:10.033190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090dce-9caa-48e5-a70e-11ec5ee5177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:10.099335 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4a090dce-9caa-48e5-a70e-11ec5ee5177f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:10.100546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4a090dce-9caa-48e5-a70e-11ec5ee5177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:10.155139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:10.196251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:10.239394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544c21a-b719-4277-9854-eef3f6982a19 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:10.240975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0544c21a-b719-4277-9854-eef3f6982a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:10.382778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544c21a-b719-4277-9854-eef3f6982a19 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:10.454446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:10.495205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f5b5e-727e-4114-838e-f7679f8f56b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:10.495827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f5b5e-727e-4114-838e-f7679f8f56b3 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:10.496366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f5b5e-727e-4114-838e-f7679f8f56b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:10.496631 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f5b5e-727e-4114-838e-f7679f8f56b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:10.498216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-252f5b5e-727e-4114-838e-f7679f8f56b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:10.585183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f5b5e-727e-4114-838e-f7679f8f56b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:10.586771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f5b5e-727e-4114-838e-f7679f8f56b3 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:10.604966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 709/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:11.501606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6736ab-5646-4f81-8cf9-e0374f6a1e76 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:11.502140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6736ab-5646-4f81-8cf9-e0374f6a1e76 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:11.502140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6736ab-5646-4f81-8cf9-e0374f6a1e76 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:11.503549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 09:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:11.508413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c330ce0-289d-4250-999b-d7475bb98c3b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:11.508811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c330ce0-289d-4250-999b-d7475bb98c3b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:11.508977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c330ce0-289d-4250-999b-d7475bb98c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:11.581788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0c330ce0-289d-4250-999b-d7475bb98c3b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:11.582976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0c330ce0-289d-4250-999b-d7475bb98c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:11.608343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:11.675980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 09:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:11.824827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6736ab-5646-4f81-8cf9-e0374f6a1e76 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:11.826403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6d6736ab-5646-4f81-8cf9-e0374f6a1e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:11.968560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6736ab-5646-4f81-8cf9-e0374f6a1e76 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:12.007117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:12.077729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeeef77-3514-4207-aafa-14fb52a33f3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:12.078349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeef77-3514-4207-aafa-14fb52a33f3e service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:12.078621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeef77-3514-4207-aafa-14fb52a33f3e service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:12.078886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeef77-3514-4207-aafa-14fb52a33f3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:12.080471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7eeeef77-3514-4207-aafa-14fb52a33f3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:12.165727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeeef77-3514-4207-aafa-14fb52a33f3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:12.167322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeeef77-3514-4207-aafa-14fb52a33f3e service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:12.184678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 709/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:13.067016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaf5e3-bb1a-4cc4-a86c-d3a5a04db8ed None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:13.067649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaf5e3-bb1a-4cc4-a86c-d3a5a04db8ed None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:13.067649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaf5e3-bb1a-4cc4-a86c-d3a5a04db8ed None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:13.068926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 09:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:13.073882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4860e-8aaf-4ca6-aaf7-bc684d66af1c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:13.074355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4860e-8aaf-4ca6-aaf7-bc684d66af1c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:13.074460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4860e-8aaf-4ca6-aaf7-bc684d66af1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:13.127570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:13.148643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e1a4860e-8aaf-4ca6-aaf7-bc684d66af1c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:13.149825 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e1a4860e-8aaf-4ca6-aaf7-bc684d66af1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:13.245326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 09:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:13.396929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaf5e3-bb1a-4cc4-a86c-d3a5a04db8ed None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:13.398137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-facaf5e3-bb1a-4cc4-a86c-d3a5a04db8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:13.540648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaf5e3-bb1a-4cc4-a86c-d3a5a04db8ed None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:13.553221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:13.656305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2094f0-9390-44ca-89b4-d06f6c2c4167 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:13.656827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2094f0-9390-44ca-89b4-d06f6c2c4167 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:13.657097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2094f0-9390-44ca-89b4-d06f6c2c4167 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:13.657365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2094f0-9390-44ca-89b4-d06f6c2c4167 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:13.658970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7d2094f0-9390-44ca-89b4-d06f6c2c4167 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:13.744511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2094f0-9390-44ca-89b4-d06f6c2c4167 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:13.746347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2094f0-9390-44ca-89b4-d06f6c2c4167 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:13.763897 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 712/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:14.621833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f0c8d-b2df-41ba-9f1c-dc17d5d9c543 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:14.622370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f0c8d-b2df-41ba-9f1c-dc17d5d9c543 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:14.622370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f0c8d-b2df-41ba-9f1c-dc17d5d9c543 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:14.623738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 09:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:14.629193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588fe6c-ece4-462a-b1b1-f994ecebcf53 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:14.629593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588fe6c-ece4-462a-b1b1-f994ecebcf53 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:14.629747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588fe6c-ece4-462a-b1b1-f994ecebcf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:14.649205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:14.706526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-2588fe6c-ece4-462a-b1b1-f994ecebcf53 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:14.707660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-2588fe6c-ece4-462a-b1b1-f994ecebcf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:14.801623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 09:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:14.949261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f0c8d-b2df-41ba-9f1c-dc17d5d9c543 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:14.950767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9d2f0c8d-b2df-41ba-9f1c-dc17d5d9c543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:15.061341 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:15.096790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f0c8d-b2df-41ba-9f1c-dc17d5d9c543 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:15.204609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5472a-64ce-4a40-bc96-1e74dc5e6ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:15.205208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5472a-64ce-4a40-bc96-1e74dc5e6ce6 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:15.205486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5472a-64ce-4a40-bc96-1e74dc5e6ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:15.205758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5472a-64ce-4a40-bc96-1e74dc5e6ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:15.207400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d3e5472a-64ce-4a40-bc96-1e74dc5e6ce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:15.281597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:15.295862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5472a-64ce-4a40-bc96-1e74dc5e6ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:15.297448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5472a-64ce-4a40-bc96-1e74dc5e6ce6 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:15.315869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 712/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:16.164410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1df3e-c6d2-4734-9065-753c2ba0795a None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:16.164983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1df3e-c6d2-4734-9065-753c2ba0795a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:16.164983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1df3e-c6d2-4734-9065-753c2ba0795a None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:16.166265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 09:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:16.171856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32344bd7-37eb-4e2a-a80b-25f6050ee2a4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:16.172258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32344bd7-37eb-4e2a-a80b-25f6050ee2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:16.172408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32344bd7-37eb-4e2a-a80b-25f6050ee2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:16.247187 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-32344bd7-37eb-4e2a-a80b-25f6050ee2a4 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:16.248388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-32344bd7-37eb-4e2a-a80b-25f6050ee2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:16.341157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 09:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:16.489915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1df3e-c6d2-4734-9065-753c2ba0795a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:16.491371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9ae1df3e-c6d2-4734-9065-753c2ba0795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:16.572425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:16.632607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1df3e-c6d2-4734-9065-753c2ba0795a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:16.861006 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491e7fe-c525-4696-8c0d-4a4c361e3c7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:16.861590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491e7fe-c525-4696-8c0d-4a4c361e3c7f service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:16.861859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491e7fe-c525-4696-8c0d-4a4c361e3c7f service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:16.862120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491e7fe-c525-4696-8c0d-4a4c361e3c7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:16.863712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b491e7fe-c525-4696-8c0d-4a4c361e3c7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:16.910748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:16.950773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491e7fe-c525-4696-8c0d-4a4c361e3c7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:16.953100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491e7fe-c525-4696-8c0d-4a4c361e3c7f service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:16.970249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 715/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:17.779678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a9ca-237b-4b2a-a67f-2ce5527836cc None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:17.780204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a9ca-237b-4b2a-a67f-2ce5527836cc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:17.780204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a9ca-237b-4b2a-a67f-2ce5527836cc None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:17.781979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 09:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:17.786819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143934d-0507-4ecb-af59-034557eee04c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:17.787287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143934d-0507-4ecb-af59-034557eee04c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:17.787405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143934d-0507-4ecb-af59-034557eee04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:17.860416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-8143934d-0507-4ecb-af59-034557eee04c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:17.861597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8143934d-0507-4ecb-af59-034557eee04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:17.953538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 09:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:18.102839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a9ca-237b-4b2a-a67f-2ce5527836cc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:18.104388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-24e2a9ca-237b-4b2a-a67f-2ce5527836cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:18.152623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:18.247654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a9ca-237b-4b2a-a67f-2ce5527836cc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:18.355814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99c452-0d75-4d56-b8e3-579023f3f38a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:18.356345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99c452-0d75-4d56-b8e3-579023f3f38a service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:18.356624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99c452-0d75-4d56-b8e3-579023f3f38a service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:18.356893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99c452-0d75-4d56-b8e3-579023f3f38a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:18.358495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8f99c452-0d75-4d56-b8e3-579023f3f38a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:18.365220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:18.446195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99c452-0d75-4d56-b8e3-579023f3f38a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:18.447795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99c452-0d75-4d56-b8e3-579023f3f38a service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:18.465689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 715/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:19.261935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e963907-8bf5-48ec-9668-d3388c4fe84a None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:19.262482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e963907-8bf5-48ec-9668-d3388c4fe84a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:19.262482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e963907-8bf5-48ec-9668-d3388c4fe84a None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:19.263905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 09:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:19.268713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6b11-a7d6-478c-a5c6-073b6a2c4261 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:19.269136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6b11-a7d6-478c-a5c6-073b6a2c4261 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:19.269293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6b11-a7d6-478c-a5c6-073b6a2c4261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:19.343315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b67b6b11-a7d6-478c-a5c6-073b6a2c4261 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:19.344545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b67b6b11-a7d6-478c-a5c6-073b6a2c4261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:19.410736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:19.440482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 09:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:19.588482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e963907-8bf5-48ec-9668-d3388c4fe84a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:19.589998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1e963907-8bf5-48ec-9668-d3388c4fe84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:19.608201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:19.734399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e963907-8bf5-48ec-9668-d3388c4fe84a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:19.818134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:19.844690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8a12f-84d8-40af-b6ee-e9a9d504f786 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:19.845211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8a12f-84d8-40af-b6ee-e9a9d504f786 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:19.845480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8a12f-84d8-40af-b6ee-e9a9d504f786 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:19.845739 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8a12f-84d8-40af-b6ee-e9a9d504f786 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:19.847324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-38e8a12f-84d8-40af-b6ee-e9a9d504f786 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:19.931445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8a12f-84d8-40af-b6ee-e9a9d504f786 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:19.932997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8a12f-84d8-40af-b6ee-e9a9d504f786 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:19.950446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 718/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:20.742667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fc180-566e-47ee-87f3-839224a149d5 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:20.743252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fc180-566e-47ee-87f3-839224a149d5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:20.743252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fc180-566e-47ee-87f3-839224a149d5 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:20.744553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 09:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:20.749702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b59443-194d-4e64-977d-f01857e467d1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:20.750091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b59443-194d-4e64-977d-f01857e467d1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:20.750236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b59443-194d-4e64-977d-f01857e467d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:20.822406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a5b59443-194d-4e64-977d-f01857e467d1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:20.823599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a5b59443-194d-4e64-977d-f01857e467d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:20.863828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:20.915909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 09:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:21.065848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fc180-566e-47ee-87f3-839224a149d5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:21.067386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d22fc180-566e-47ee-87f3-839224a149d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:21.208170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fc180-566e-47ee-87f3-839224a149d5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:21.260307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:21.322879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54137d-3ca1-4358-8d1b-2863df304ed0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:21.323430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54137d-3ca1-4358-8d1b-2863df304ed0 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:21.323706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54137d-3ca1-4358-8d1b-2863df304ed0 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:21.323975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54137d-3ca1-4358-8d1b-2863df304ed0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:21.325639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2d54137d-3ca1-4358-8d1b-2863df304ed0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:21.412837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54137d-3ca1-4358-8d1b-2863df304ed0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:21.414419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54137d-3ca1-4358-8d1b-2863df304ed0 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:21.432005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 718/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:22.231775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb7c23-ceb5-4d1d-8d16-bd2011cac40e None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:22.232305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb7c23-ceb5-4d1d-8d16-bd2011cac40e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:22.232305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb7c23-ceb5-4d1d-8d16-bd2011cac40e None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:22.234148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 09:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:22.238932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46f4cf-cae5-4dc5-96a4-8b5e7f829c87 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:22.239409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46f4cf-cae5-4dc5-96a4-8b5e7f829c87 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:22.239513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46f4cf-cae5-4dc5-96a4-8b5e7f829c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:22.313633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fe46f4cf-cae5-4dc5-96a4-8b5e7f829c87 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:22.314835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fe46f4cf-cae5-4dc5-96a4-8b5e7f829c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:22.316890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:22.411298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 09:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:22.559331 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb7c23-ceb5-4d1d-8d16-bd2011cac40e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:22.560851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-83bb7c23-ceb5-4d1d-8d16-bd2011cac40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:22.701131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb7c23-ceb5-4d1d-8d16-bd2011cac40e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:22.727378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:22.809498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914019e-2a56-442f-a865-3479dd57e320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:22.810016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914019e-2a56-442f-a865-3479dd57e320 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:22.810282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914019e-2a56-442f-a865-3479dd57e320 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:22.810547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914019e-2a56-442f-a865-3479dd57e320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:22.812126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-2914019e-2a56-442f-a865-3479dd57e320 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:22.896478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914019e-2a56-442f-a865-3479dd57e320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:22.898039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914019e-2a56-442f-a865-3479dd57e320 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:22.915622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 721/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:24.813900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf897a25-409f-478f-a762-ba3859a3ab9c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:24.814425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf897a25-409f-478f-a762-ba3859a3ab9c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:24.814425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf897a25-409f-478f-a762-ba3859a3ab9c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:24.815764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:24.820877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c9932-afa5-4879-8d5c-da7421ac0ee4 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:24.821270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c9932-afa5-4879-8d5c-da7421ac0ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:24.821416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c9932-afa5-4879-8d5c-da7421ac0ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:24.861090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:24.886325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c29c9932-afa5-4879-8d5c-da7421ac0ee4 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:24.887531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c29c9932-afa5-4879-8d5c-da7421ac0ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:24.979481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:25.020100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf897a25-409f-478f-a762-ba3859a3ab9c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:25.021674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bf897a25-409f-478f-a762-ba3859a3ab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:25.122611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf897a25-409f-478f-a762-ba3859a3ab9c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:25.153664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:25.194618 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95db339-bdba-4abd-8c03-7c754fd69b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:25.195234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95db339-bdba-4abd-8c03-7c754fd69b81 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:25.195509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95db339-bdba-4abd-8c03-7c754fd69b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:25.195773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95db339-bdba-4abd-8c03-7c754fd69b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:25.197334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f95db339-bdba-4abd-8c03-7c754fd69b81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:25.301919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95db339-bdba-4abd-8c03-7c754fd69b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:25.303368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95db339-bdba-4abd-8c03-7c754fd69b81 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:25.321020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 721/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:27.451021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd75c8-8c8d-412f-9b1d-0d7409fa2a7b None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:27.451488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd75c8-8c8d-412f-9b1d-0d7409fa2a7b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:27.451605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd75c8-8c8d-412f-9b1d-0d7409fa2a7b None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:27.452879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:27.458521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549289-0c67-4565-b60e-f24e7ee21115 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:27.458980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549289-0c67-4565-b60e-f24e7ee21115 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:27.459091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549289-0c67-4565-b60e-f24e7ee21115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:27.523573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-71549289-0c67-4565-b60e-f24e7ee21115 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:27.524698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-71549289-0c67-4565-b60e-f24e7ee21115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:27.537175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:27.619896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:57:27.658309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd75c8-8c8d-412f-9b1d-0d7409fa2a7b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:27.659900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f4fd75c8-8c8d-412f-9b1d-0d7409fa2a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:27.761947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd75c8-8c8d-412f-9b1d-0d7409fa2a7b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:27.774644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:27.834467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1735d7-29dc-4994-b510-52c81eafee9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:27.835089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1735d7-29dc-4994-b510-52c81eafee9c service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:27.835366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1735d7-29dc-4994-b510-52c81eafee9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:27.835633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1735d7-29dc-4994-b510-52c81eafee9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:27.837209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3d1735d7-29dc-4994-b510-52c81eafee9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:27.943009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1735d7-29dc-4994-b510-52c81eafee9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:27.944426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1735d7-29dc-4994-b510-52c81eafee9c service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:27.961614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 724/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:30.269135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069145c-1c5a-4dc9-943e-06257018ef61 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:30.269663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069145c-1c5a-4dc9-943e-06257018ef61 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:30.269663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069145c-1c5a-4dc9-943e-06257018ef61 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:30.271037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:30.276239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60969ec1-955a-4568-b953-78575373001a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:30.276663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60969ec1-955a-4568-b953-78575373001a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:30.276777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60969ec1-955a-4568-b953-78575373001a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:30.324046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:30.341757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-60969ec1-955a-4568-b953-78575373001a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:30.342870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-60969ec1-955a-4568-b953-78575373001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:30.438045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:30.476423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069145c-1c5a-4dc9-943e-06257018ef61 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:30.477978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7069145c-1c5a-4dc9-943e-06257018ef61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:30.579239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069145c-1c5a-4dc9-943e-06257018ef61 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:30.618290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:30.650519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c140652-e053-4f3f-89af-fc66f596b87f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:30.651139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c140652-e053-4f3f-89af-fc66f596b87f service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:30.651414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c140652-e053-4f3f-89af-fc66f596b87f service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:30.651679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c140652-e053-4f3f-89af-fc66f596b87f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:30.653244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8c140652-e053-4f3f-89af-fc66f596b87f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:30.762902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c140652-e053-4f3f-89af-fc66f596b87f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:30.764351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c140652-e053-4f3f-89af-fc66f596b87f service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:30.782573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 724/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:32.865789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469ef9-498d-432f-b42b-24123c3b0858 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:32.866343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469ef9-498d-432f-b42b-24123c3b0858 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:32.866343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469ef9-498d-432f-b42b-24123c3b0858 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:32.867731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:32.873064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb16ea-3bab-4032-92b1-209fa78a6b48 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:32.873462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb16ea-3bab-4032-92b1-209fa78a6b48 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:32.873606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb16ea-3bab-4032-92b1-209fa78a6b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:32.937044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-eaeb16ea-3bab-4032-92b1-209fa78a6b48 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:32.938736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-eaeb16ea-3bab-4032-92b1-209fa78a6b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:32.959117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:33.035221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:33.074321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71469ef9-498d-432f-b42b-24123c3b0858 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:33.075877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-71469ef9-498d-432f-b42b-24123c3b0858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:33.177594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71469ef9-498d-432f-b42b-24123c3b0858 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:33.197286 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:33.251075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2466acb-adc9-4334-9bc4-9c9bb853099e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:33.251680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2466acb-adc9-4334-9bc4-9c9bb853099e service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:33.251951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2466acb-adc9-4334-9bc4-9c9bb853099e service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:33.252217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2466acb-adc9-4334-9bc4-9c9bb853099e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:33.253800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e2466acb-adc9-4334-9bc4-9c9bb853099e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:33.360171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2466acb-adc9-4334-9bc4-9c9bb853099e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:33.361591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2466acb-adc9-4334-9bc4-9c9bb853099e service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:33.379701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 727/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:35.426724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4d8cd-24fc-4deb-a8d3-8e358e471bdd None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:35.427196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4d8cd-24fc-4deb-a8d3-8e358e471bdd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:35.427317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4d8cd-24fc-4deb-a8d3-8e358e471bdd None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:35.428656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:35.433871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e11d34-eca0-4258-95f8-0e850c55290b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:35.434261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e11d34-eca0-4258-95f8-0e850c55290b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:35.434413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e11d34-eca0-4258-95f8-0e850c55290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:35.496233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:35.499161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-97e11d34-eca0-4258-95f8-0e850c55290b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:35.500890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-97e11d34-eca0-4258-95f8-0e850c55290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:35.595267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:35.632821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4d8cd-24fc-4deb-a8d3-8e358e471bdd None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:35.634421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-28c4d8cd-24fc-4deb-a8d3-8e358e471bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:35.735669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4d8cd-24fc-4deb-a8d3-8e358e471bdd None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:35.783408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:35.807349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf5036-9dff-4913-8526-a7bb98defba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:35.807946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf5036-9dff-4913-8526-a7bb98defba0 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:35.808221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf5036-9dff-4913-8526-a7bb98defba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:35.808507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf5036-9dff-4913-8526-a7bb98defba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:35.810062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4cbf5036-9dff-4913-8526-a7bb98defba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:35.915271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf5036-9dff-4913-8526-a7bb98defba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:35.916687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf5036-9dff-4913-8526-a7bb98defba0 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:35.934508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 727/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:38.008835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bef17-66ca-4803-afae-95f2eb87801d None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:38.009378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bef17-66ca-4803-afae-95f2eb87801d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:38.009378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bef17-66ca-4803-afae-95f2eb87801d None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:38.010705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:38.016402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b3071-9147-4fd5-9b4a-7dbd6e150cf5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:38.016778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b3071-9147-4fd5-9b4a-7dbd6e150cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:38.016950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b3071-9147-4fd5-9b4a-7dbd6e150cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:38.080489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fc3b3071-9147-4fd5-9b4a-7dbd6e150cf5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:38.081681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fc3b3071-9147-4fd5-9b4a-7dbd6e150cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:38.109515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:38.177943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:38.217039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bef17-66ca-4803-afae-95f2eb87801d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:38.218569 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a46bef17-66ca-4803-afae-95f2eb87801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:38.319638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bef17-66ca-4803-afae-95f2eb87801d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:38.346444 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:38.392184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deeaaee-0409-4fa2-bcf0-c5af83090b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:38.392783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deeaaee-0409-4fa2-bcf0-c5af83090b48 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:38.393059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deeaaee-0409-4fa2-bcf0-c5af83090b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:38.393326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deeaaee-0409-4fa2-bcf0-c5af83090b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:38.394909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4deeaaee-0409-4fa2-bcf0-c5af83090b48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:38.501122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deeaaee-0409-4fa2-bcf0-c5af83090b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:38.502550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deeaaee-0409-4fa2-bcf0-c5af83090b48 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:38.519700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 730/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:40.567694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3372f-6213-4433-983f-1d195fb6405c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:40.568100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3372f-6213-4433-983f-1d195fb6405c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:40.568262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3372f-6213-4433-983f-1d195fb6405c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:40.569605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:40.574894 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880646c-29a9-4db9-814f-27cc60ee19a9 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:40.575357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880646c-29a9-4db9-814f-27cc60ee19a9 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:40.575482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880646c-29a9-4db9-814f-27cc60ee19a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:40.640385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-5880646c-29a9-4db9-814f-27cc60ee19a9 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:40.641590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5880646c-29a9-4db9-814f-27cc60ee19a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:40.649573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:40.734016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:40.771864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3372f-6213-4433-983f-1d195fb6405c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:40.773410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d0c3372f-6213-4433-983f-1d195fb6405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:40.873646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3372f-6213-4433-983f-1d195fb6405c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:40.928507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:40.946240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4611-6751-431e-8b8f-385c6b363d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:40.946834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4611-6751-431e-8b8f-385c6b363d7b service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:40.947134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4611-6751-431e-8b8f-385c6b363d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:40.947401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4611-6751-431e-8b8f-385c6b363d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:40.948966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-95db4611-6751-431e-8b8f-385c6b363d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:41.054724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db4611-6751-431e-8b8f-385c6b363d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:41.056183 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db4611-6751-431e-8b8f-385c6b363d7b service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:41.073515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 730/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:43.089086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10696197-84f4-46dc-ad50-8ed40a119295 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:43.089647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10696197-84f4-46dc-ad50-8ed40a119295 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:43.089647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10696197-84f4-46dc-ad50-8ed40a119295 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:43.090935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:43.096823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5c38c-32d9-479d-938e-3a60e91aff2b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:43.097278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5c38c-32d9-479d-938e-3a60e91aff2b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:43.097381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5c38c-32d9-479d-938e-3a60e91aff2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:43.165013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0de5c38c-32d9-479d-938e-3a60e91aff2b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:43.166209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0de5c38c-32d9-479d-938e-3a60e91aff2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:43.205594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:43.264093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:43.301516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10696197-84f4-46dc-ad50-8ed40a119295 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:43.303112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-10696197-84f4-46dc-ad50-8ed40a119295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:43.404877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10696197-84f4-46dc-ad50-8ed40a119295 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:43.441554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:43.483008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e363d5c-132b-494d-a4f8-c6f0d65be79f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:43.483612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e363d5c-132b-494d-a4f8-c6f0d65be79f service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:43.483887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e363d5c-132b-494d-a4f8-c6f0d65be79f service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:43.484154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e363d5c-132b-494d-a4f8-c6f0d65be79f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:43.485757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8e363d5c-132b-494d-a4f8-c6f0d65be79f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:43.594584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e363d5c-132b-494d-a4f8-c6f0d65be79f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:43.596041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e363d5c-132b-494d-a4f8-c6f0d65be79f service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:43.613941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 733/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:45.715276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf841a8-a61f-4cd6-b66c-9f1bf6f64b7e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:45.715845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf841a8-a61f-4cd6-b66c-9f1bf6f64b7e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:45.715845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf841a8-a61f-4cd6-b66c-9f1bf6f64b7e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:45.717080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:45.722844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af561e-f25f-4670-acab-a17fe0127df7 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:45.723306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af561e-f25f-4670-acab-a17fe0127df7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:45.723430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af561e-f25f-4670-acab-a17fe0127df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:45.787509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-33af561e-f25f-4670-acab-a17fe0127df7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:45.788697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-33af561e-f25f-4670-acab-a17fe0127df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:45.806531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:45.890399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:45.927482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf841a8-a61f-4cd6-b66c-9f1bf6f64b7e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:45.929008 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9cf841a8-a61f-4cd6-b66c-9f1bf6f64b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:46.029821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf841a8-a61f-4cd6-b66c-9f1bf6f64b7e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:46.095158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:46.102512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a2411-7aa8-461b-8e98-c412d8956ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:46.103149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a2411-7aa8-461b-8e98-c412d8956ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:46.103430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a2411-7aa8-461b-8e98-c412d8956ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:46.103701 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a2411-7aa8-461b-8e98-c412d8956ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:46.105338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-706a2411-7aa8-461b-8e98-c412d8956ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:46.212448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a2411-7aa8-461b-8e98-c412d8956ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:46.213875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a2411-7aa8-461b-8e98-c412d8956ea2 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:46.231318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 733/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:48.265545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7761f-3150-49b0-8b85-b7a9113f91fb None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:48.266115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7761f-3150-49b0-8b85-b7a9113f91fb None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:48.266115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7761f-3150-49b0-8b85-b7a9113f91fb None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:48.267454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 09:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:57:48.272596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a545-1159-4269-8a77-7b514469918b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:48.273011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a545-1159-4269-8a77-7b514469918b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:48.273145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a545-1159-4269-8a77-7b514469918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:48.336621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7c91a545-1159-4269-8a77-7b514469918b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:57:48.337793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7c91a545-1159-4269-8a77-7b514469918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:48.384088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:48.431969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 09:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:57:48.467859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7761f-3150-49b0-8b85-b7a9113f91fb None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:57:48.469899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0bc7761f-3150-49b0-8b85-b7a9113f91fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:48.572847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7761f-3150-49b0-8b85-b7a9113f91fb None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:57:48.619489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:48.645102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3495c-daf7-44ac-858e-b9f53484e1c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:57:48.645728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3495c-daf7-44ac-858e-b9f53484e1c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:57:48.646007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3495c-daf7-44ac-858e-b9f53484e1c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:57:48.646275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3495c-daf7-44ac-858e-b9f53484e1c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:57:48.647895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-afd3495c-daf7-44ac-858e-b9f53484e1c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:57:48.755545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3495c-daf7-44ac-858e-b9f53484e1c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:57:48.756969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3495c-daf7-44ac-858e-b9f53484e1c5 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:57:48.774870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 736/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:57:58.430156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=22 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:57:58.772638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=66 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:08.123162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ff723-3ee3-41bf-94f9-95d348696c31 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:08.123735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ff723-3ee3-41bf-94f9-95d348696c31 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:08.123735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ff723-3ee3-41bf-94f9-95d348696c31 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:08.125044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 735/1471] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:58:08.130286 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be695ad6-58ff-43db-9e2f-ce7355cc8093 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:08.130705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be695ad6-58ff-43db-9e2f-ce7355cc8093 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:08.130834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be695ad6-58ff-43db-9e2f-ce7355cc8093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:08.199443 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-be695ad6-58ff-43db-9e2f-ce7355cc8093 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:58:08.200622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-be695ad6-58ff-43db-9e2f-ce7355cc8093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:08.294504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 737/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:58:08.376952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8df759-41d3-4c44-977b-276dd293d016 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:08.377179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8df759-41d3-4c44-977b-276dd293d016 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:08.377446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8df759-41d3-4c44-977b-276dd293d016 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:08.378764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 736/1473] 10.222.0.20 () {54 vars in 967 bytes} [Wed Aug 10 09:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:58:08.383677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038fe6-8b96-49ea-b7ec-02df9204cc7e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:08.383971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038fe6-8b96-49ea-b7ec-02df9204cc7e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:08.384244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038fe6-8b96-49ea-b7ec-02df9204cc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:08.459540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-1a038fe6-8b96-49ea-b7ec-02df9204cc7e None None] MFA Rules not processed for user `b8f456f0d15b4d068cfcd6476563247f`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:58:08.460741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1a038fe6-8b96-49ea-b7ec-02df9204cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:08.480829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:08.535535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 738/1474] 10.222.0.20 () {58 vars in 1064 bytes} [Wed Aug 10 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:58:08.543319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8df759-41d3-4c44-977b-276dd293d016 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:58:08.544825 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3f8df759-41d3-4c44-977b-276dd293d016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:08.646012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8df759-41d3-4c44-977b-276dd293d016 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:58:08.716910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa1e63-ffa2-4da1-95a8-ad18b3af028d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8f456f0d15b4d068cfcd6476563247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:58:08.717537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa1e63-ffa2-4da1-95a8-ad18b3af028d service glance] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:08.717818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa1e63-ffa2-4da1-95a8-ad18b3af028d service glance] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:08.718092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa1e63-ffa2-4da1-95a8-ad18b3af028d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:08.719756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a2aa1e63-ffa2-4da1-95a8-ad18b3af028d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:08.775420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:08.829488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa1e63-ffa2-4da1-95a8-ad18b3af028d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:58:08.830919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa1e63-ffa2-4da1-95a8-ad18b3af028d service glance] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:58:08.849826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 737/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:58:10.932447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd51d2-fc98-44de-b5c6-2ff69f107d7d None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:10.933015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd51d2-fc98-44de-b5c6-2ff69f107d7d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:10.933015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd51d2-fc98-44de-b5c6-2ff69f107d7d None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:10.934317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 739/1476] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:58:10.939625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf2d27-566b-4a75-b830-0eb8a07b8b70 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:10.940012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf2d27-566b-4a75-b830-0eb8a07b8b70 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:10.940173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf2d27-566b-4a75-b830-0eb8a07b8b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:11.004426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d1cf2d27-566b-4a75-b830-0eb8a07b8b70 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:58:11.005625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d1cf2d27-566b-4a75-b830-0eb8a07b8b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:11.099007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 738/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:58:11.121836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd51d2-fc98-44de-b5c6-2ff69f107d7d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:58:11.123619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-97dd51d2-fc98-44de-b5c6-2ff69f107d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:11.245072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd51d2-fc98-44de-b5c6-2ff69f107d7d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:58:11.292460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:11.336625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f2e70-2273-4daf-bb31-2abcdcf3fa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:58:11.337221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f2e70-2273-4daf-bb31-2abcdcf3fa99 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:11.337498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f2e70-2273-4daf-bb31-2abcdcf3fa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:11.337766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f2e70-2273-4daf-bb31-2abcdcf3fa99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:11.338396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f2e70-2273-4daf-bb31-2abcdcf3fa99 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:58:11.340132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WARNING py.warnings [None req-ea0f2e70-2273-4daf-bb31-2abcdcf3fa99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 09:58:11.340132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: warnings.warn(msg) Aug 10 09:58:11.340132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: Aug 10 09:58:11.341262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f2e70-2273-4daf-bb31-2abcdcf3fa99 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:58:11.359878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 740/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 10 09:58:11 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:58:13.368124 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03b43c-8957-4ba2-b3fe-e7de977afd6a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:13.368666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03b43c-8957-4ba2-b3fe-e7de977afd6a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:13.368666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03b43c-8957-4ba2-b3fe-e7de977afd6a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:13.370001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 739/1479] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:58:13.375834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd83e30-14fc-4209-a1f9-76fa654cde41 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:13.376301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd83e30-14fc-4209-a1f9-76fa654cde41 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:13.376414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd83e30-14fc-4209-a1f9-76fa654cde41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:13.441575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-3dd83e30-14fc-4209-a1f9-76fa654cde41 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:58:13.442764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3dd83e30-14fc-4209-a1f9-76fa654cde41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:13.537282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 741/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:58:13.580883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03b43c-8957-4ba2-b3fe-e7de977afd6a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:58:13.582391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4e03b43c-8957-4ba2-b3fe-e7de977afd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:13.592505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:13.724490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03b43c-8957-4ba2-b3fe-e7de977afd6a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:58:13.802324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:13.834666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c2805-6a2d-4581-a1f6-17b46cb57dea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:58:13.835291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c2805-6a2d-4581-a1f6-17b46cb57dea service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:13.835571 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c2805-6a2d-4581-a1f6-17b46cb57dea service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:13.835840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c2805-6a2d-4581-a1f6-17b46cb57dea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:13.837416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b00c2805-6a2d-4581-a1f6-17b46cb57dea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:13.924562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c2805-6a2d-4581-a1f6-17b46cb57dea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:58:13.926176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c2805-6a2d-4581-a1f6-17b46cb57dea service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:58:13.943701 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 740/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:58:16.131949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036818a-7ae2-442f-90df-cc7eb00192d2 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:16.132553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036818a-7ae2-442f-90df-cc7eb00192d2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:16.132553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036818a-7ae2-442f-90df-cc7eb00192d2 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:16.133806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 742/1482] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:58:16.139211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04a5a6-cc11-4e7e-a164-93c25d969217 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:16.139640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04a5a6-cc11-4e7e-a164-93c25d969217 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:16.139747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04a5a6-cc11-4e7e-a164-93c25d969217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:16.207058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5d04a5a6-cc11-4e7e-a164-93c25d969217 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:58:16.208277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5d04a5a6-cc11-4e7e-a164-93c25d969217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:16.253906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:16.303639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 741/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:58:16.346209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036818a-7ae2-442f-90df-cc7eb00192d2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:58:16.347762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7036818a-7ae2-442f-90df-cc7eb00192d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:16.370062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:16.449725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036818a-7ae2-442f-90df-cc7eb00192d2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:58:16.520478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe80c17-e0e7-4a41-95db-47d6e5038403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:58:16.521071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe80c17-e0e7-4a41-95db-47d6e5038403 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:16.521343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe80c17-e0e7-4a41-95db-47d6e5038403 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:16.521609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe80c17-e0e7-4a41-95db-47d6e5038403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:16.523209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-dbe80c17-e0e7-4a41-95db-47d6e5038403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:16.594442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:16.640712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe80c17-e0e7-4a41-95db-47d6e5038403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:58:16.642912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe80c17-e0e7-4a41-95db-47d6e5038403 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:58:16.660833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 743/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:58:18.704831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769576-fd3b-45b6-a5c1-74c22fde3132 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:18.705353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769576-fd3b-45b6-a5c1-74c22fde3132 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:18.705353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769576-fd3b-45b6-a5c1-74c22fde3132 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:18.706632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 742/1485] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:58:18.711697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5c19b-7bca-4fb5-840b-c3a09702999d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:18.712114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5c19b-7bca-4fb5-840b-c3a09702999d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:18.712282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5c19b-7bca-4fb5-840b-c3a09702999d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:18.779588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-45e5c19b-7bca-4fb5-840b-c3a09702999d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:58:18.780773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-45e5c19b-7bca-4fb5-840b-c3a09702999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:18.874892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 744/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:58:18.913222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5769576-fd3b-45b6-a5c1-74c22fde3132 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:58:18.914766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c5769576-fd3b-45b6-a5c1-74c22fde3132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:19.015988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5769576-fd3b-45b6-a5c1-74c22fde3132 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:58:19.083959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:19.086059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec293d-a9d7-4318-bef8-af539bf40470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:58:19.086898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec293d-a9d7-4318-bef8-af539bf40470 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:19.087270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec293d-a9d7-4318-bef8-af539bf40470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:19.087761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec293d-a9d7-4318-bef8-af539bf40470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:19.089515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-74ec293d-a9d7-4318-bef8-af539bf40470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:19.196754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec293d-a9d7-4318-bef8-af539bf40470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:58:19.198207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec293d-a9d7-4318-bef8-af539bf40470 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:58:19.216951 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 743/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:58:21.358361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e53c95-9aa8-408d-aabd-0bf7195695dd None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:21.358925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e53c95-9aa8-408d-aabd-0bf7195695dd None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:21.358925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e53c95-9aa8-408d-aabd-0bf7195695dd None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:21.360309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 745/1488] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:58:21.365328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aec14-2acf-4b01-abe5-af8e4c8cb766 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:21.365753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aec14-2acf-4b01-abe5-af8e4c8cb766 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:21.365861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aec14-2acf-4b01-abe5-af8e4c8cb766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:21.432343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4d6aec14-2acf-4b01-abe5-af8e4c8cb766 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:58:21.433527 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4d6aec14-2acf-4b01-abe5-af8e4c8cb766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:21.484261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:21.528073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 744/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:58:21.567993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e53c95-9aa8-408d-aabd-0bf7195695dd None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:58:21.569508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-57e53c95-9aa8-408d-aabd-0bf7195695dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:21.604709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:21.672851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e53c95-9aa8-408d-aabd-0bf7195695dd None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:58:21.742984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba424c-1d6d-437f-9400-fe63e4b3bbbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:58:21.743582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba424c-1d6d-437f-9400-fe63e4b3bbbc service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:21.743856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba424c-1d6d-437f-9400-fe63e4b3bbbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:21.744122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba424c-1d6d-437f-9400-fe63e4b3bbbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:21.745688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c3ba424c-1d6d-437f-9400-fe63e4b3bbbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:21.826608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:21.857244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba424c-1d6d-437f-9400-fe63e4b3bbbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:58:21.858678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba424c-1d6d-437f-9400-fe63e4b3bbbc service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:58:21.877906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 746/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:58:24.047951 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd30e51-deca-4d75-b6d9-459f672840d4 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:24.048467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd30e51-deca-4d75-b6d9-459f672840d4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:24.048467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd30e51-deca-4d75-b6d9-459f672840d4 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:24.049850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 745/1491] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 10 09:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:58:24.054998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884766a-f486-440e-a1c6-c2ac4f1094af None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:24.055435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884766a-f486-440e-a1c6-c2ac4f1094af None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:24.055540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884766a-f486-440e-a1c6-c2ac4f1094af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:24.119522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-7884766a-f486-440e-a1c6-c2ac4f1094af None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:58:24.120705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7884766a-f486-440e-a1c6-c2ac4f1094af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:24.214195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 747/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 10 09:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:58:24.254469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd30e51-deca-4d75-b6d9-459f672840d4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:58:24.256086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ecd30e51-deca-4d75-b6d9-459f672840d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:24.355377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd30e51-deca-4d75-b6d9-459f672840d4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:58:24.428075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ec998-92dd-4bf6-a21c-90f66c609ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:58:24.428670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ec998-92dd-4bf6-a21c-90f66c609ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:58:24.428945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ec998-92dd-4bf6-a21c-90f66c609ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:58:24.429212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ec998-92dd-4bf6-a21c-90f66c609ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:58:24.430839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ca9ec998-92dd-4bf6-a21c-90f66c609ec9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:58:24.439805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:24.538381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ec998-92dd-4bf6-a21c-90f66c609ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:58:24.539832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ec998-92dd-4bf6-a21c-90f66c609ec9 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:58:24.559255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 746/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:58:34.212413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=82 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:58:34.556924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=50 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:33.926111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2ff02-dcea-499e-a062-addeadcd8bdd None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:33.926697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2ff02-dcea-499e-a062-addeadcd8bdd None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:33.926697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2ff02-dcea-499e-a062-addeadcd8bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:34.002387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-16d2ff02-dcea-499e-a062-addeadcd8bdd None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:59:34.003610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-16d2ff02-dcea-499e-a062-addeadcd8bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:34.100649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 748/1494] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 09:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:59:34.108961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae215d7-5480-4cbc-a6a1-ee2f59189be9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:34.109463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae215d7-5480-4cbc-a6a1-ee2f59189be9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:34.109579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae215d7-5480-4cbc-a6a1-ee2f59189be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:34.182179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fae215d7-5480-4cbc-a6a1-ee2f59189be9 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:59:34.183313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fae215d7-5480-4cbc-a6a1-ee2f59189be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:34.277119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 747/1495] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 09:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:59:34.281732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2ff02-dcea-499e-a062-addeadcd8bdd None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:34.282978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-16d2ff02-dcea-499e-a062-addeadcd8bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:34.296379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2ff02-dcea-499e-a062-addeadcd8bdd None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:34.359122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:34.393042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:34.393644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:34.393921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:34.394190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:34.394824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:59:34.395151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:34.396161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WARNING py.warnings [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 09:59:34.396161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: warnings.warn(msg) Aug 10 09:59:34.396161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: Aug 10 09:59:34.397049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:34.408278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 749/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 09:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:59:34.416603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae215d7-5480-4cbc-a6a1-ee2f59189be9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:34.425673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae215d7-5480-4cbc-a6a1-ee2f59189be9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:34.485742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:34.519716 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:34.520313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:34.520586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:34.520852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:34.521472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:34.523240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 09:59:34.523240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Aug 10 09:59:34.523240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: Aug 10 09:59:34.524416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:34.579714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 748/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 09:59:34 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:59:34.583949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:34.585549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:34.619115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfefa9-3dc4-408f-93ec-18726926ff4e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:34.718636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:34.719271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:34.719544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:34.719812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:34.720436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:34.721464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WARNING py.warnings [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 09:59:34.721464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: warnings.warn(msg) Aug 10 09:59:34.721464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: Aug 10 09:59:34.722691 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:34.764345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 750/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 09:59:34 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:59:34.768399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:34.769888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:34.829149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54c087-96da-492c-a4c0-6c3809a96710 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:34.910867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:34.922791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:34.923421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:34.923699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:34.923969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:34.924595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:34.925619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 09:59:34.925619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Aug 10 09:59:34.925619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: Aug 10 09:59:34.926494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:34.938613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 749/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 09:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:59:34.942540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:34.944091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:34.955195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354996ee-8b6e-406a-9c1c-c573c5198f5f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:34.979751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:35.049942 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4b20e-5828-420d-8ada-0fa7d47feea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:35.050530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4b20e-5828-420d-8ada-0fa7d47feea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:35.050800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4b20e-5828-420d-8ada-0fa7d47feea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:35.051093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4b20e-5828-420d-8ada-0fa7d47feea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:35.051719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4b20e-5828-420d-8ada-0fa7d47feea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:35.052709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WARNING py.warnings [None req-6de4b20e-5828-420d-8ada-0fa7d47feea1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 09:59:35.052709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: warnings.warn(msg) Aug 10 09:59:35.052709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: Aug 10 09:59:35.053579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4b20e-5828-420d-8ada-0fa7d47feea1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:35.065484 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 751/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 09:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:59:35.069688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:35.071261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:35.103766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1eeda-57e9-416c-9f69-90feeded4fef admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:35.196266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e55f4-c919-4601-8836-1b7b905f4e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:35.196940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e55f4-c919-4601-8836-1b7b905f4e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:35.197186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e55f4-c919-4601-8836-1b7b905f4e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:35.197476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e55f4-c919-4601-8836-1b7b905f4e66 admin admin] PATH_INFO: `/v3/projects/751453aefc454858b95a0da4967d7a47/users/ca4d0369d8de43afa8e88c18a2cc26da/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:35.220460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e55f4-c919-4601-8836-1b7b905f4e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751453aefc454858b95a0da4967d7a47, user_id=ca4d0369d8de43afa8e88c18a2cc26da, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:35.221470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-2f3e55f4-c919-4601-8836-1b7b905f4e66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 09:59:35.221470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Aug 10 09:59:35.221470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: Aug 10 09:59:35.223410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e55f4-c919-4601-8836-1b7b905f4e66 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:35.248811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 750/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 09:59:35 2022] PUT /identity/v3/projects/751453aefc454858b95a0da4967d7a47/users/ca4d0369d8de43afa8e88c18a2cc26da/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:59:35.261175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b7b44-eab8-435c-8448-dc48a39efeae None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:35.261466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b7b44-eab8-435c-8448-dc48a39efeae None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:35.261744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b7b44-eab8-435c-8448-dc48a39efeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:35.317244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-9c3b7b44-eab8-435c-8448-dc48a39efeae None None] MFA Rules not processed for user `ca4d0369d8de43afa8e88c18a2cc26da`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:59:35.318374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9c3b7b44-eab8-435c-8448-dc48a39efeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:35.395248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 752/1502] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 09:59:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:59:35.409143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6412f-4da1-4644-a2c6-fa6dd22e2c30 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:35.409433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6412f-4da1-4644-a2c6-fa6dd22e2c30 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:35.409711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6412f-4da1-4644-a2c6-fa6dd22e2c30 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:35.411106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 751/1503] 10.222.0.20 () {54 vars in 964 bytes} [Wed Aug 10 09:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:59:35.415962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5161ec-b85c-4caa-9399-53acafd7d050 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:35.416247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5161ec-b85c-4caa-9399-53acafd7d050 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:35.416516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5161ec-b85c-4caa-9399-53acafd7d050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:35.447094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:35.490649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ea5161ec-b85c-4caa-9399-53acafd7d050 None None] MFA Rules not processed for user `f1e3ba8d0c7d4f8190cd0b5b057f5638`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:59:35.491868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ea5161ec-b85c-4caa-9399-53acafd7d050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:35.565035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 753/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Aug 10 09:59:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:59:35.572844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6412f-4da1-4644-a2c6-fa6dd22e2c30 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:35.574283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dea6412f-4da1-4644-a2c6-fa6dd22e2c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:35.623817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:35.676640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6412f-4da1-4644-a2c6-fa6dd22e2c30 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:35.746815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:35.747429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:35.747703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:35.747969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:35.750085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:35.845760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:35.847142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:35.850563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:35.865014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 752/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:59:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:59:35.909845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183457f-fcca-422b-bb6c-c5c91654c6ec None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:35.910137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183457f-fcca-422b-bb6c-c5c91654c6ec None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:35.910412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183457f-fcca-422b-bb6c-c5c91654c6ec None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:35.911692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 754/1506] 10.222.0.20 () {52 vars in 830 bytes} [Wed Aug 10 09:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 09:59:35.915241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:35.916456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:36.018656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65c259-95eb-41cc-8cd8-20b1e7d3a195 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:36.110293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:36.110843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:36.111150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:36.111453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] PATH_INFO: `/v3/users/ca4d0369d8de43afa8e88c18a2cc26da` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:36.124016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca4d0369d8de43afa8e88c18a2cc26da)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:36.125034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 09:59:36.125034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Aug 10 09:59:36.125034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: Aug 10 09:59:36.126247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:36.135722 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:36.191585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca4d0369d8de43afa8e88c18a2cc26da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:36.205187 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca4d0369d8de43afa8e88c18a2cc26da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:36.226856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca4d0369d8de43afa8e88c18a2cc26da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:36.227467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca4d0369d8de43afa8e88c18a2cc26da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:36.227895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca4d0369d8de43afa8e88c18a2cc26da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:36.241504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 753/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 09:59:35 2022] DELETE /identity/v3/users/ca4d0369d8de43afa8e88c18a2cc26da => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:59:36.245898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cc7ce-fa15-4342-9949-fb4e96ebc5cd None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:36.246407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cc7ce-fa15-4342-9949-fb4e96ebc5cd None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:36.246524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cc7ce-fa15-4342-9949-fb4e96ebc5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:36.302177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-725cc7ce-fa15-4342-9949-fb4e96ebc5cd None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:59:36.303397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-725cc7ce-fa15-4342-9949-fb4e96ebc5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:36.376481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:36.388896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 755/1508] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 09:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:59:36.398606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:36.399907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:36.499059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb45a52-1e06-4a37-8874-1ce01be25d60 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:36.551693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:36.571188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e35690-4bab-4fa7-af7d-9fc45f6a6f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:36.571795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e35690-4bab-4fa7-af7d-9fc45f6a6f4b service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:36.572066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e35690-4bab-4fa7-af7d-9fc45f6a6f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:36.572333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e35690-4bab-4fa7-af7d-9fc45f6a6f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:36.573997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-87e35690-4bab-4fa7-af7d-9fc45f6a6f4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:36.679836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e35690-4bab-4fa7-af7d-9fc45f6a6f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:36.681236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e35690-4bab-4fa7-af7d-9fc45f6a6f4b service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:36.699224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 754/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 09:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 09:59:37.028988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cc7ce-fa15-4342-9949-fb4e96ebc5cd None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:59:37.031491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-725cc7ce-fa15-4342-9949-fb4e96ebc5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:59:37.043563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cc7ce-fa15-4342-9949-fb4e96ebc5cd None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:59:37.136123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:59:37.136749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:59:37.137021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:59:37.137285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] PATH_INFO: `/v3/projects/751453aefc454858b95a0da4967d7a47` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:59:37.147489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] RBAC: Authorizing `identity:delete_project(project_id=751453aefc454858b95a0da4967d7a47)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:59:37.148514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WARNING py.warnings [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 09:59:37.148514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: warnings.warn(msg) Aug 10 09:59:37.148514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: Aug 10 09:59:37.149702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:59:37.207979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751453aefc454858b95a0da4967d7a47', 'request_id': 'req-10ac42f9-546a-4393-ac1c-73281ef0a34f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:37.220766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751453aefc454858b95a0da4967d7a47', 'request_id': 'req-10ac42f9-546a-4393-ac1c-73281ef0a34f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:37.229583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751453aefc454858b95a0da4967d7a47', 'request_id': 'req-10ac42f9-546a-4393-ac1c-73281ef0a34f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:37.251802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] The token cache is being invalidate because project 751453aefc454858b95a0da4967d7a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:59:37.252373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:37.252482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-10ac42f9-546a-4393-ac1c-73281ef0a34f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:59:37.253853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 756/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 09:59:37 2022] DELETE /identity/v3/projects/751453aefc454858b95a0da4967d7a47 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:59:46.697046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=62 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:59:47.249868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:01:05.701942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6decf6-958f-438a-a113-f71d4a72e23d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:05.702501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6decf6-958f-438a-a113-f71d4a72e23d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:05.702501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6decf6-958f-438a-a113-f71d4a72e23d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:05.703805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 10:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:01:05.708803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b7ec-7412-4d20-9258-b1b16e43e8a1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:05.709254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b7ec-7412-4d20-9258-b1b16e43e8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:05.709369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b7ec-7412-4d20-9258-b1b16e43e8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:05.789059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-bb39b7ec-7412-4d20-9258-b1b16e43e8a1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:05.790242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bb39b7ec-7412-4d20-9258-b1b16e43e8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:05.890351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 10:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:01:06.043917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6decf6-958f-438a-a113-f71d4a72e23d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:01:06.046015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0b6decf6-958f-438a-a113-f71d4a72e23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:06.164059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6decf6-958f-438a-a113-f71d4a72e23d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:01:06.169440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd836cb6-fdec-4d1b-9336-3f35de0961f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:01:06.170059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd836cb6-fdec-4d1b-9336-3f35de0961f2 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:06.170334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd836cb6-fdec-4d1b-9336-3f35de0961f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:06.170621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd836cb6-fdec-4d1b-9336-3f35de0961f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:06.178380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd836cb6-fdec-4d1b-9336-3f35de0961f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:01:06.179966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd836cb6-fdec-4d1b-9336-3f35de0961f2 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:01:06.189070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 756/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:01:07.021134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bc736-1a33-4200-b50e-46729bd832a2 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:07.021720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bc736-1a33-4200-b50e-46729bd832a2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:07.021720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bc736-1a33-4200-b50e-46729bd832a2 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:07.023048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 10:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:01:07.027997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e938b3d-2937-4b63-9757-31d324d9d22a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:07.028445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e938b3d-2937-4b63-9757-31d324d9d22a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:07.028558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e938b3d-2937-4b63-9757-31d324d9d22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:07.047813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6e938b3d-2937-4b63-9757-31d324d9d22a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:07.048989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6e938b3d-2937-4b63-9757-31d324d9d22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:07.056932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 10:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:01:07.207594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bc736-1a33-4200-b50e-46729bd832a2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:01:07.221968 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bc736-1a33-4200-b50e-46729bd832a2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:01:07.227334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a28dbd-9e92-451d-b29b-529aecfe20bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:01:07.227918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a28dbd-9e92-451d-b29b-529aecfe20bd service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:07.228196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a28dbd-9e92-451d-b29b-529aecfe20bd service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:07.228466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a28dbd-9e92-451d-b29b-529aecfe20bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:07.236904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a28dbd-9e92-451d-b29b-529aecfe20bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:01:07.238474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a28dbd-9e92-451d-b29b-529aecfe20bd service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:01:07.247659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 759/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:01:07.835747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea0785-454f-4d1c-bbe1-6ca0008bafd3 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:07.836031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea0785-454f-4d1c-bbe1-6ca0008bafd3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:07.836309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea0785-454f-4d1c-bbe1-6ca0008bafd3 None None] PATH_INFO: `/v3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:07.837635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 758/1517] 10.222.0.17 () {54 vars in 925 bytes} [Wed Aug 10 10:01:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:01:07.844116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015f41f-ab86-4c79-8169-6a66e8b131ef None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:07.844404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015f41f-ab86-4c79-8169-6a66e8b131ef None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:07.844680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015f41f-ab86-4c79-8169-6a66e8b131ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:07.902588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c015f41f-ab86-4c79-8169-6a66e8b131ef None None] MFA Rules not processed for user `d3d7b3cb498a4da18e203456e09b59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:07.903812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c015f41f-ab86-4c79-8169-6a66e8b131ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:07.920009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:01:07.974372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 760/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 10 10:01:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:01:11.790876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06302-2d9e-42de-b16f-5a8d87548f57 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:11.791423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06302-2d9e-42de-b16f-5a8d87548f57 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:11.791532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06302-2d9e-42de-b16f-5a8d87548f57 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:11.792782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 759/1519] 10.222.0.20 () {54 vars in 961 bytes} [Wed Aug 10 10:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:01:11.797797 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbe661-1c28-47c7-a5f9-1b5a276a6c2c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:11.798244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbe661-1c28-47c7-a5f9-1b5a276a6c2c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:11.798374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbe661-1c28-47c7-a5f9-1b5a276a6c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:11.819321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-30dbe661-1c28-47c7-a5f9-1b5a276a6c2c None None] MFA Rules not processed for user `d3d7b3cb498a4da18e203456e09b59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:11.820493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-30dbe661-1c28-47c7-a5f9-1b5a276a6c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:11.829462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 761/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Aug 10 10:01:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:01:11.836859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb06302-2d9e-42de-b16f-5a8d87548f57 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:01:11.849878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb06302-2d9e-42de-b16f-5a8d87548f57 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:01:11.855173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6bb6e-ce4a-4db5-9e88-584e47a7ce52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:01:11.855756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6bb6e-ce4a-4db5-9e88-584e47a7ce52 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:11.856038 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6bb6e-ce4a-4db5-9e88-584e47a7ce52 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:11.856312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6bb6e-ce4a-4db5-9e88-584e47a7ce52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:11.864306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6bb6e-ce4a-4db5-9e88-584e47a7ce52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:01:11.865867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6bb6e-ce4a-4db5-9e88-584e47a7ce52 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:01:11.875980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 760/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:01:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:01:18.473075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec82b60-4e8d-4ad2-a6b7-065ad03c62b4 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:18.473601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec82b60-4e8d-4ad2-a6b7-065ad03c62b4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:18.473721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec82b60-4e8d-4ad2-a6b7-065ad03c62b4 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:18.475059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 10:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:01:18.479688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8b86-760b-4d1e-aa75-4e7547c07665 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:18.480080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8b86-760b-4d1e-aa75-4e7547c07665 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:18.480233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8b86-760b-4d1e-aa75-4e7547c07665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:18.500014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3a5b8b86-760b-4d1e-aa75-4e7547c07665 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:18.501205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3a5b8b86-760b-4d1e-aa75-4e7547c07665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:18.508929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 10:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:01:18.661718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec82b60-4e8d-4ad2-a6b7-065ad03c62b4 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:01:18.676812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec82b60-4e8d-4ad2-a6b7-065ad03c62b4 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:01:18.682223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41751e04-fc68-403f-a5cc-35f85c895903 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:01:18.682811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41751e04-fc68-403f-a5cc-35f85c895903 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:18.683130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41751e04-fc68-403f-a5cc-35f85c895903 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:18.683402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41751e04-fc68-403f-a5cc-35f85c895903 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:18.691340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41751e04-fc68-403f-a5cc-35f85c895903 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:01:18.692907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41751e04-fc68-403f-a5cc-35f85c895903 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:01:18.702113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 763/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:01:19.567660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7c76f-9e45-40e5-86ad-3aacce07db09 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:19.568238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7c76f-9e45-40e5-86ad-3aacce07db09 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:19.568238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7c76f-9e45-40e5-86ad-3aacce07db09 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:19.569554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 10:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:01:19.574480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337e494-0696-4261-8bc2-7b18ab64e2de None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:19.574868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337e494-0696-4261-8bc2-7b18ab64e2de None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:19.575049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337e494-0696-4261-8bc2-7b18ab64e2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:19.594757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0337e494-0696-4261-8bc2-7b18ab64e2de None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:19.595967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0337e494-0696-4261-8bc2-7b18ab64e2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:19.603760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:01:19.753290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7c76f-9e45-40e5-86ad-3aacce07db09 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:01:19.767607 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7c76f-9e45-40e5-86ad-3aacce07db09 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:01:19.772859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f9785-4ea7-4efb-b55d-e82763585a66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:01:19.773444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f9785-4ea7-4efb-b55d-e82763585a66 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:19.773721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f9785-4ea7-4efb-b55d-e82763585a66 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:19.773990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f9785-4ea7-4efb-b55d-e82763585a66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:19.781801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f9785-4ea7-4efb-b55d-e82763585a66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:01:19.783393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f9785-4ea7-4efb-b55d-e82763585a66 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:01:19.792675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 763/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:01:23.640300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233065b-6c88-4c9f-b344-8554b626744c None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:23.640830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233065b-6c88-4c9f-b344-8554b626744c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:23.640953 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233065b-6c88-4c9f-b344-8554b626744c None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:23.642258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 10:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:01:23.647449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0209a-eefa-41e9-b198-997bf2cca16d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:23.647857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0209a-eefa-41e9-b198-997bf2cca16d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:23.647987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0209a-eefa-41e9-b198-997bf2cca16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:23.667320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a2a0209a-eefa-41e9-b198-997bf2cca16d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:23.668419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a2a0209a-eefa-41e9-b198-997bf2cca16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:23.676648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:01:23.748859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233065b-6c88-4c9f-b344-8554b626744c None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:01:23.750828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b233065b-6c88-4c9f-b344-8554b626744c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:23.826921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233065b-6c88-4c9f-b344-8554b626744c None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:01:23.830516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f92aa-4393-4fe0-9f2b-60a932420f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:01:23.831140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f92aa-4393-4fe0-9f2b-60a932420f3c service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:23.831418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f92aa-4393-4fe0-9f2b-60a932420f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:23.831690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f92aa-4393-4fe0-9f2b-60a932420f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:23.840311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f92aa-4393-4fe0-9f2b-60a932420f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:01:23.841711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f92aa-4393-4fe0-9f2b-60a932420f3c service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:01:23.851315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 766/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:01:25.845124 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d7142-f455-4015-9397-fc483ee2a47b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:25.845685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d7142-f455-4015-9397-fc483ee2a47b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:25.845685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d7142-f455-4015-9397-fc483ee2a47b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:25.847032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 10:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:01:25.852164 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d7c50-57cc-4f2f-8f1c-db04c2bc150d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:25.852555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d7c50-57cc-4f2f-8f1c-db04c2bc150d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:25.852706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d7c50-57cc-4f2f-8f1c-db04c2bc150d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:25.872296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-034d7c50-57cc-4f2f-8f1c-db04c2bc150d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:25.873460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-034d7c50-57cc-4f2f-8f1c-db04c2bc150d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:25.881365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 10:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:01:25.916346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d7142-f455-4015-9397-fc483ee2a47b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:01:25.927420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d7142-f455-4015-9397-fc483ee2a47b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:01:25.930837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532033a-a6a6-4fb2-a39e-376d0f5e8973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:01:25.931449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532033a-a6a6-4fb2-a39e-376d0f5e8973 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:25.931724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532033a-a6a6-4fb2-a39e-376d0f5e8973 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:25.931994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532033a-a6a6-4fb2-a39e-376d0f5e8973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:25.940718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532033a-a6a6-4fb2-a39e-376d0f5e8973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:01:25.942140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532033a-a6a6-4fb2-a39e-376d0f5e8973 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:01:25.951574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 766/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:01:29.438840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62f780-8705-4ca2-b223-1c5c70088ada None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:29.439343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62f780-8705-4ca2-b223-1c5c70088ada None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:29.439456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62f780-8705-4ca2-b223-1c5c70088ada None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:29.440758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 10:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:01:29.445720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a93b7-0284-4503-b6c6-a563dd779ea5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:29.446162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a93b7-0284-4503-b6c6-a563dd779ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:29.446292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a93b7-0284-4503-b6c6-a563dd779ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:29.454121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=81 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:01:29.466600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9f5a93b7-0284-4503-b6c6-a563dd779ea5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:29.468426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f5a93b7-0284-4503-b6c6-a563dd779ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:29.476083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 10:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:01:29.775626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62f780-8705-4ca2-b223-1c5c70088ada None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:01:29.777275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bb62f780-8705-4ca2-b223-1c5c70088ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:29.840527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:01:29.853634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62f780-8705-4ca2-b223-1c5c70088ada None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:01:29.857111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b83a4-1f7e-4c8c-95b7-6b0f251cba1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:01:29.857699 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b83a4-1f7e-4c8c-95b7-6b0f251cba1b service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:29.857979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b83a4-1f7e-4c8c-95b7-6b0f251cba1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:29.858273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b83a4-1f7e-4c8c-95b7-6b0f251cba1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:29.867426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b83a4-1f7e-4c8c-95b7-6b0f251cba1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:01:29.868835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b83a4-1f7e-4c8c-95b7-6b0f251cba1b service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:01:29.878758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 769/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:01:31.877087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15b78d-a9e5-4ba5-9a3f-a6a90642b81d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:31.877648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15b78d-a9e5-4ba5-9a3f-a6a90642b81d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:31.877648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15b78d-a9e5-4ba5-9a3f-a6a90642b81d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:31.879039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 10:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:01:31.884156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f995b0-3bf1-44f7-986c-fdf890e14207 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:31.884587 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f995b0-3bf1-44f7-986c-fdf890e14207 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:31.884692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f995b0-3bf1-44f7-986c-fdf890e14207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:31.904295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f6f995b0-3bf1-44f7-986c-fdf890e14207 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:01:31.905517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f6f995b0-3bf1-44f7-986c-fdf890e14207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:31.913317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 10:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:01:32.214596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15b78d-a9e5-4ba5-9a3f-a6a90642b81d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:01:32.216290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-be15b78d-a9e5-4ba5-9a3f-a6a90642b81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:01:32.228899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15b78d-a9e5-4ba5-9a3f-a6a90642b81d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:01:32.232372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50514a-079b-4c47-be38-7d9044ad6ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:01:32.232955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50514a-079b-4c47-be38-7d9044ad6ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:01:32.233229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50514a-079b-4c47-be38-7d9044ad6ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:01:32.234159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50514a-079b-4c47-be38-7d9044ad6ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:01:32.242729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50514a-079b-4c47-be38-7d9044ad6ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:01:32.244160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50514a-079b-4c47-be38-7d9044ad6ec2 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:01:32.254303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 769/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:01:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:01:41.899485 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=11 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:01:42.249251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=8 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:54.895558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9cf81-32cc-4715-9a4f-d98c7180f5a7 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:54.898945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9cf81-32cc-4715-9a4f-d98c7180f5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:54.898945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9cf81-32cc-4715-9a4f-d98c7180f5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:54.938291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-94c9cf81-32cc-4715-9a4f-d98c7180f5a7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:54.939556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-94c9cf81-32cc-4715-9a4f-d98c7180f5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:54.942442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0615a-6bcd-4a65-9824-112ae3c0d3e3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:54.943493 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0615a-6bcd-4a65-9824-112ae3c0d3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:54.944117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0615a-6bcd-4a65-9824-112ae3c0d3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:54.958765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 771/1540] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:02:54.984513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-23e0615a-6bcd-4a65-9824-112ae3c0d3e3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:54.986071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-23e0615a-6bcd-4a65-9824-112ae3c0d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:54.996004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b265ea-d7b0-428d-9b72-5e61b7d4c258 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:54.996417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b265ea-d7b0-428d-9b72-5e61b7d4c258 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:54.996715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b265ea-d7b0-428d-9b72-5e61b7d4c258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:54.998419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 770/1541] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.007281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b60ac-b6bf-4c17-a37a-49bb87c4eef5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.008005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b60ac-b6bf-4c17-a37a-49bb87c4eef5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.008599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b60ac-b6bf-4c17-a37a-49bb87c4eef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.056130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-68b265ea-d7b0-428d-9b72-5e61b7d4c258 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.057353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-68b265ea-d7b0-428d-9b72-5e61b7d4c258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.063879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f21b60ac-b6bf-4c17-a37a-49bb87c4eef5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.066613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f21b60ac-b6bf-4c17-a37a-49bb87c4eef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.077444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 771/1542] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:02:55.083631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f42e4-f5e6-4f6a-8cba-b2de881e1a57 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.084263 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f42e4-f5e6-4f6a-8cba-b2de881e1a57 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.084757 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f42e4-f5e6-4f6a-8cba-b2de881e1a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.098980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 772/1543] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.098980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b265ea-d7b0-428d-9b72-5e61b7d4c258 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.117699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e09f42e4-f5e6-4f6a-8cba-b2de881e1a57 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.119516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e09f42e4-f5e6-4f6a-8cba-b2de881e1a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.125575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b265ea-d7b0-428d-9b72-5e61b7d4c258 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.139802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 772/1544] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.140720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f42e4-f5e6-4f6a-8cba-b2de881e1a57 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.159808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53c4c9-319f-4c79-a337-e11cc2d2b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.160075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f42e4-f5e6-4f6a-8cba-b2de881e1a57 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.160454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53c4c9-319f-4c79-a337-e11cc2d2b37f admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.160891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53c4c9-319f-4c79-a337-e11cc2d2b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.161175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53c4c9-319f-4c79-a337-e11cc2d2b37f admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.161854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53c4c9-319f-4c79-a337-e11cc2d2b37f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:02:55.162182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53c4c9-319f-4c79-a337-e11cc2d2b37f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.163868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53c4c9-319f-4c79-a337-e11cc2d2b37f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.175863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd23962-5131-4c47-a32d-38870e72d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.176508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd23962-5131-4c47-a32d-38870e72d489 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.176794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd23962-5131-4c47-a32d-38870e72d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.177069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd23962-5131-4c47-a32d-38870e72d489 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.177739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd23962-5131-4c47-a32d-38870e72d489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:02:55.178045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd23962-5131-4c47-a32d-38870e72d489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.179117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-cdd23962-5131-4c47-a32d-38870e72d489 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 10:02:55.179117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Aug 10 10:02:55.179117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: Aug 10 10:02:55.180091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd23962-5131-4c47-a32d-38870e72d489 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.193576 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 773/1545] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:02:55.194863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc683aab-2052-4a71-a3eb-1cd9c4df2e7f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.195179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc683aab-2052-4a71-a3eb-1cd9c4df2e7f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.195447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc683aab-2052-4a71-a3eb-1cd9c4df2e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.207078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 773/1546] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:02:55.208330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11799e97-705d-48ee-a41c-7539cc061703 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.208630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11799e97-705d-48ee-a41c-7539cc061703 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.208911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11799e97-705d-48ee-a41c-7539cc061703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.249019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-fc683aab-2052-4a71-a3eb-1cd9c4df2e7f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.250415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-fc683aab-2052-4a71-a3eb-1cd9c4df2e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.253048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-11799e97-705d-48ee-a41c-7539cc061703 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.254237 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-11799e97-705d-48ee-a41c-7539cc061703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.270070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 774/1547] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.271313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc683aab-2052-4a71-a3eb-1cd9c4df2e7f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.281297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 774/1548] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.283355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11799e97-705d-48ee-a41c-7539cc061703 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.293059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc683aab-2052-4a71-a3eb-1cd9c4df2e7f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.297443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11799e97-705d-48ee-a41c-7539cc061703 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.304190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122b1db-418d-4574-b22b-2182d9309e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.304807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122b1db-418d-4574-b22b-2182d9309e4b admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.305086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122b1db-418d-4574-b22b-2182d9309e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.305358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122b1db-418d-4574-b22b-2182d9309e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.305986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122b1db-418d-4574-b22b-2182d9309e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.307075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WARNING py.warnings [None req-f122b1db-418d-4574-b22b-2182d9309e4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 10:02:55.307075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: warnings.warn(msg) Aug 10 10:02:55.307075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: Aug 10 10:02:55.309518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122b1db-418d-4574-b22b-2182d9309e4b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.312189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcef5a-3d13-4b2e-9326-59713d6571f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.313590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcef5a-3d13-4b2e-9326-59713d6571f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.314241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcef5a-3d13-4b2e-9326-59713d6571f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.314784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcef5a-3d13-4b2e-9326-59713d6571f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.315798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcef5a-3d13-4b2e-9326-59713d6571f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.319125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcef5a-3d13-4b2e-9326-59713d6571f2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.353295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 775/1549] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:55.355089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72a404-797f-43a2-a4b0-59c0e74c0fdc None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.355577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72a404-797f-43a2-a4b0-59c0e74c0fdc None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.356066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 775/1550] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:55.356431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72a404-797f-43a2-a4b0-59c0e74c0fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.360517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b7d57-89c7-49f2-beb8-4d44fc2cad64 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.360517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b7d57-89c7-49f2-beb8-4d44fc2cad64 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.360517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b7d57-89c7-49f2-beb8-4d44fc2cad64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.380554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-169b7d57-89c7-49f2-beb8-4d44fc2cad64 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.381743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-169b7d57-89c7-49f2-beb8-4d44fc2cad64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.384065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-9f72a404-797f-43a2-a4b0-59c0e74c0fdc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.385408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9f72a404-797f-43a2-a4b0-59c0e74c0fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.444117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 776/1551] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.445363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7287f77-531c-4e9b-af27-2ed74fcc6a26 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.445653 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7287f77-531c-4e9b-af27-2ed74fcc6a26 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.445929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7287f77-531c-4e9b-af27-2ed74fcc6a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.447381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 776/1552] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.450829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72a404-797f-43a2-a4b0-59c0e74c0fdc None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.460946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72a404-797f-43a2-a4b0-59c0e74c0fdc None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.465451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f7287f77-531c-4e9b-af27-2ed74fcc6a26 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.465598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370127d-c215-4f3b-9cb6-9ca57a1948c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.466044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370127d-c215-4f3b-9cb6-9ca57a1948c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.466317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370127d-c215-4f3b-9cb6-9ca57a1948c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.466588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370127d-c215-4f3b-9cb6-9ca57a1948c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.466741 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f7287f77-531c-4e9b-af27-2ed74fcc6a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.467259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370127d-c215-4f3b-9cb6-9ca57a1948c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:02:55.468210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370127d-c215-4f3b-9cb6-9ca57a1948c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.469779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370127d-c215-4f3b-9cb6-9ca57a1948c9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.475148 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 777/1553] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.475895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7287f77-531c-4e9b-af27-2ed74fcc6a26 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.485250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7287f77-531c-4e9b-af27-2ed74fcc6a26 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.491950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 777/1554] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:02:55.491950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370127d-c215-4f3b-9cb6-9ca57a1948c9 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.493928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7888f21-30ff-4a78-9196-88df5bfd3332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.494681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7888f21-30ff-4a78-9196-88df5bfd3332 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.495029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7888f21-30ff-4a78-9196-88df5bfd3332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.495310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7888f21-30ff-4a78-9196-88df5bfd3332 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.495928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7888f21-30ff-4a78-9196-88df5bfd3332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.496966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-d7888f21-30ff-4a78-9196-88df5bfd3332 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 10:02:55.496966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Aug 10 10:02:55.496966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: Aug 10 10:02:55.498162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7888f21-30ff-4a78-9196-88df5bfd3332 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.501330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370127d-c215-4f3b-9cb6-9ca57a1948c9 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.505759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e950c8-33bd-476a-8979-295501c2de37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.506331 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e950c8-33bd-476a-8979-295501c2de37 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.506601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e950c8-33bd-476a-8979-295501c2de37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.506866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e950c8-33bd-476a-8979-295501c2de37 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.507568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e950c8-33bd-476a-8979-295501c2de37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.509424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e950c8-33bd-476a-8979-295501c2de37 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.520493 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 778/1555] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:55.521780 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fce4e3-72a3-4ffe-bf1d-68bbc78264cc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.522042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fce4e3-72a3-4ffe-bf1d-68bbc78264cc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.522312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fce4e3-72a3-4ffe-bf1d-68bbc78264cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.538848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 778/1556] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:55.538848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b7a16-2eca-449c-a6a4-9075587a0d27 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.538848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b7a16-2eca-449c-a6a4-9075587a0d27 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.538848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b7a16-2eca-449c-a6a4-9075587a0d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.544772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a2fce4e3-72a3-4ffe-bf1d-68bbc78264cc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.546067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a2fce4e3-72a3-4ffe-bf1d-68bbc78264cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.553948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 779/1557] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.554738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fce4e3-72a3-4ffe-bf1d-68bbc78264cc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.555598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-612b7a16-2eca-449c-a6a4-9075587a0d27 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.556746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-612b7a16-2eca-449c-a6a4-9075587a0d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.564363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fce4e3-72a3-4ffe-bf1d-68bbc78264cc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.567275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 779/1558] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.568513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71461755-8866-472d-a7c7-2f655f066a0f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.568795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71461755-8866-472d-a7c7-2f655f066a0f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.569010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c759b-d423-4ee6-8c0d-abaa760450c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.569141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71461755-8866-472d-a7c7-2f655f066a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.569610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c759b-d423-4ee6-8c0d-abaa760450c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.569889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c759b-d423-4ee6-8c0d-abaa760450c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.570162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c759b-d423-4ee6-8c0d-abaa760450c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.570802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c759b-d423-4ee6-8c0d-abaa760450c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:02:55.571283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c759b-d423-4ee6-8c0d-abaa760450c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.572782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c759b-d423-4ee6-8c0d-abaa760450c6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.593750 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 780/1559] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:02:55.593750 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc20d9-525c-43d7-b5bc-5d9a3bb88b3e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.593750 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc20d9-525c-43d7-b5bc-5d9a3bb88b3e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.593750 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc20d9-525c-43d7-b5bc-5d9a3bb88b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.594233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-71461755-8866-472d-a7c7-2f655f066a0f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.594233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-71461755-8866-472d-a7c7-2f655f066a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.600508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 780/1560] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.601130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71461755-8866-472d-a7c7-2f655f066a0f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.607908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8bcc20d9-525c-43d7-b5bc-5d9a3bb88b3e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.609116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8bcc20d9-525c-43d7-b5bc-5d9a3bb88b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.614437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71461755-8866-472d-a7c7-2f655f066a0f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.616822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 781/1561] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.617565 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc20d9-525c-43d7-b5bc-5d9a3bb88b3e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.622287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677303b7-c5d0-4dfc-b48d-19515c396b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.622910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677303b7-c5d0-4dfc-b48d-19515c396b0a admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.623223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677303b7-c5d0-4dfc-b48d-19515c396b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.623799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677303b7-c5d0-4dfc-b48d-19515c396b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.624502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677303b7-c5d0-4dfc-b48d-19515c396b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.626655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677303b7-c5d0-4dfc-b48d-19515c396b0a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.627645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc20d9-525c-43d7-b5bc-5d9a3bb88b3e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.632727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97b15-a5fb-4fb7-8a1f-7545a3a9ad93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.633315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97b15-a5fb-4fb7-8a1f-7545a3a9ad93 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.633590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97b15-a5fb-4fb7-8a1f-7545a3a9ad93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.633856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97b15-a5fb-4fb7-8a1f-7545a3a9ad93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.634488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a97b15-a5fb-4fb7-8a1f-7545a3a9ad93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.636084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a97b15-a5fb-4fb7-8a1f-7545a3a9ad93 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.650614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 782/1562] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:55.650614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97b15-a5fb-4fb7-8a1f-7545a3a9ad93 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.660530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 781/1563] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:55.660704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97b15-a5fb-4fb7-8a1f-7545a3a9ad93 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.660821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f0da3-359c-486f-99d2-aa150278016d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.660821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f0da3-359c-486f-99d2-aa150278016d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.660821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f0da3-359c-486f-99d2-aa150278016d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.677713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c44f0da3-359c-486f-99d2-aa150278016d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.678911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c44f0da3-359c-486f-99d2-aa150278016d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.692614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:55.705935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:55.720213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8cbb6-f19a-4ee7-bf72-573a69a7defc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.721180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8cbb6-f19a-4ee7-bf72-573a69a7defc admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.721489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8cbb6-f19a-4ee7-bf72-573a69a7defc admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.721760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8cbb6-f19a-4ee7-bf72-573a69a7defc admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.722704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8cbb6-f19a-4ee7-bf72-573a69a7defc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.724370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8cbb6-f19a-4ee7-bf72-573a69a7defc admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.734227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 783/1564] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:55.735820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8711-2362-4828-a922-f0b5af843239 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.736107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8711-2362-4828-a922-f0b5af843239 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.736384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8711-2362-4828-a922-f0b5af843239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.740918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 782/1565] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.741899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f0da3-359c-486f-99d2-aa150278016d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.754515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f0da3-359c-486f-99d2-aa150278016d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.756577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-550e8711-2362-4828-a922-f0b5af843239 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:55.757777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-550e8711-2362-4828-a922-f0b5af843239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:55.759142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4620083-c7de-4b22-a485-7579ecac57c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.759736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4620083-c7de-4b22-a485-7579ecac57c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.760012 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4620083-c7de-4b22-a485-7579ecac57c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.760280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4620083-c7de-4b22-a485-7579ecac57c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.760891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4620083-c7de-4b22-a485-7579ecac57c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.762796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4620083-c7de-4b22-a485-7579ecac57c3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.766622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 784/1566] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:55.767500 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e8711-2362-4828-a922-f0b5af843239 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.777803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e8711-2362-4828-a922-f0b5af843239 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.782945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9910c9-86fb-4b99-aeaf-05c5373d59fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.783556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9910c9-86fb-4b99-aeaf-05c5373d59fb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.783831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9910c9-86fb-4b99-aeaf-05c5373d59fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.784140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9910c9-86fb-4b99-aeaf-05c5373d59fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.784836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9910c9-86fb-4b99-aeaf-05c5373d59fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:02:55.785136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9910c9-86fb-4b99-aeaf-05c5373d59fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.786706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9910c9-86fb-4b99-aeaf-05c5373d59fb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.790946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 783/1567] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:55.791810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4620083-c7de-4b22-a485-7579ecac57c3 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.806311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 785/1568] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:02:55.806311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9910c9-86fb-4b99-aeaf-05c5373d59fb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.809489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9910c9-86fb-4b99-aeaf-05c5373d59fb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.858724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4620083-c7de-4b22-a485-7579ecac57c3 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.863315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b395a2e-29f5-45a0-b333-aa3775345f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.863910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b395a2e-29f5-45a0-b333-aa3775345f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.864189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b395a2e-29f5-45a0-b333-aa3775345f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.864464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b395a2e-29f5-45a0-b333-aa3775345f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.864768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec7991-4b30-4e9c-97d3-fed242bf3a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.865096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b395a2e-29f5-45a0-b333-aa3775345f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:02:55.865363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec7991-4b30-4e9c-97d3-fed242bf3a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.865480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b395a2e-29f5-45a0-b333-aa3775345f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.865641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec7991-4b30-4e9c-97d3-fed242bf3a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.865913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec7991-4b30-4e9c-97d3-fed242bf3a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.866541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec7991-4b30-4e9c-97d3-fed242bf3a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.866980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b395a2e-29f5-45a0-b333-aa3775345f4a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.868122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec7991-4b30-4e9c-97d3-fed242bf3a62 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.878634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 786/1569] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:55.885682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 784/1570] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:02:55.885822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec7991-4b30-4e9c-97d3-fed242bf3a62 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.885942 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b395a2e-29f5-45a0-b333-aa3775345f4a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.895105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec7991-4b30-4e9c-97d3-fed242bf3a62 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.895365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b395a2e-29f5-45a0-b333-aa3775345f4a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.899633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee814c-b421-486e-8298-42428130b79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.899960 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40c9a3-3c51-461a-b312-72ca4b9394ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.900453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee814c-b421-486e-8298-42428130b79b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.900453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee814c-b421-486e-8298-42428130b79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.900640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40c9a3-3c51-461a-b312-72ca4b9394ee admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.900739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee814c-b421-486e-8298-42428130b79b admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.900833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40c9a3-3c51-461a-b312-72ca4b9394ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.901082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40c9a3-3c51-461a-b312-72ca4b9394ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.901326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee814c-b421-486e-8298-42428130b79b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.901708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40c9a3-3c51-461a-b312-72ca4b9394ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.903226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee814c-b421-486e-8298-42428130b79b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.903359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40c9a3-3c51-461a-b312-72ca4b9394ee admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.914100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 785/1571] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:55.914920 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40c9a3-3c51-461a-b312-72ca4b9394ee admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.925618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40c9a3-3c51-461a-b312-72ca4b9394ee admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.929239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 787/1572] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:55.929914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee814c-b421-486e-8298-42428130b79b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.933620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c94650-5649-4318-82b3-df5b4b1a81f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.934226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c94650-5649-4318-82b3-df5b4b1a81f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.934535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c94650-5649-4318-82b3-df5b4b1a81f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.934807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c94650-5649-4318-82b3-df5b4b1a81f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.935479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c94650-5649-4318-82b3-df5b4b1a81f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:02:55.935780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c94650-5649-4318-82b3-df5b4b1a81f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.937902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c94650-5649-4318-82b3-df5b4b1a81f4 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.940055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee814c-b421-486e-8298-42428130b79b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.944592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c17ac4-3467-4905-b136-9d6e65b1f171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.945235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c17ac4-3467-4905-b136-9d6e65b1f171 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.945418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c17ac4-3467-4905-b136-9d6e65b1f171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.945697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c17ac4-3467-4905-b136-9d6e65b1f171 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.946330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c17ac4-3467-4905-b136-9d6e65b1f171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:02:55.946627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c17ac4-3467-4905-b136-9d6e65b1f171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.948351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c17ac4-3467-4905-b136-9d6e65b1f171 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.950859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 786/1573] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:02:55.951756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c94650-5649-4318-82b3-df5b4b1a81f4 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.969107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 788/1574] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:02:55.969312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c94650-5649-4318-82b3-df5b4b1a81f4 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.969408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c17ac4-3467-4905-b136-9d6e65b1f171 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:55.969541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06523725-047a-49e7-a3f5-12420348c61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.969541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06523725-047a-49e7-a3f5-12420348c61d admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.969541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06523725-047a-49e7-a3f5-12420348c61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.969541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06523725-047a-49e7-a3f5-12420348c61d admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.969924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06523725-047a-49e7-a3f5-12420348c61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.970427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06523725-047a-49e7-a3f5-12420348c61d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.972958 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c17ac4-3467-4905-b136-9d6e65b1f171 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:55.977486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436aef8-5ced-483c-96d2-f584b5a1fca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:55.978254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436aef8-5ced-483c-96d2-f584b5a1fca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:55.978381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436aef8-5ced-483c-96d2-f584b5a1fca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:55.978629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436aef8-5ced-483c-96d2-f584b5a1fca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:55.979282 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436aef8-5ced-483c-96d2-f584b5a1fca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:55.981371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436aef8-5ced-483c-96d2-f584b5a1fca2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:55.993174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 787/1575] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/users => generated 278 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:55.993966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06523725-047a-49e7-a3f5-12420348c61d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.005034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06523725-047a-49e7-a3f5-12420348c61d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.009664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f7879-9588-4ff4-94cc-cff92fc7948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.010090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 789/1576] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:55 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:56.013702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f7879-9588-4ff4-94cc-cff92fc7948a admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.013702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f7879-9588-4ff4-94cc-cff92fc7948a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.013896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436aef8-5ced-483c-96d2-f584b5a1fca2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.013982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f7879-9588-4ff4-94cc-cff92fc7948a admin admin] PATH_INFO: `/v3/projects/229ece368fc1458ba1c6cd94cb28f87a/users/0d53857c736c4a75abe0d4979517bc58/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.027770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f7879-9588-4ff4-94cc-cff92fc7948a admin admin] RBAC: Authorizing `identity:create_grant(project_id=229ece368fc1458ba1c6cd94cb28f87a, user_id=0d53857c736c4a75abe0d4979517bc58, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.028967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WARNING py.warnings [None req-d16f7879-9588-4ff4-94cc-cff92fc7948a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 10:02:56.028967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: warnings.warn(msg) Aug 10 10:02:56.028967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: Aug 10 10:02:56.031016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f7879-9588-4ff4-94cc-cff92fc7948a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.044172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 788/1577] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:55 2022] PUT /identity/v3/projects/229ece368fc1458ba1c6cd94cb28f87a/users/0d53857c736c4a75abe0d4979517bc58/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:56.048496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f7879-9588-4ff4-94cc-cff92fc7948a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.070732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:56.074710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436aef8-5ced-483c-96d2-f584b5a1fca2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.089321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:56.110724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f7879-9588-4ff4-94cc-cff92fc7948a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.115352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68636ede-ab03-4936-b495-cb3641a38496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.116015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68636ede-ab03-4936-b495-cb3641a38496 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.116274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68636ede-ab03-4936-b495-cb3641a38496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.116543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68636ede-ab03-4936-b495-cb3641a38496 admin admin] PATH_INFO: `/v3/projects/99a927c2d1514107b2da0d6d34b35fdf/users/1e81eef05a4a40cbaab2a3b975f8d583/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.131676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68636ede-ab03-4936-b495-cb3641a38496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a927c2d1514107b2da0d6d34b35fdf, user_id=1e81eef05a4a40cbaab2a3b975f8d583, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.132689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa520-c9db-4bed-91a7-cd763679ecb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.133295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa520-c9db-4bed-91a7-cd763679ecb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.133576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa520-c9db-4bed-91a7-cd763679ecb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.133848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa520-c9db-4bed-91a7-cd763679ecb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.134378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68636ede-ab03-4936-b495-cb3641a38496 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.134472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fa520-c9db-4bed-91a7-cd763679ecb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.136359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fa520-c9db-4bed-91a7-cd763679ecb3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.146697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 789/1578] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:56 2022] PUT /identity/v3/projects/99a927c2d1514107b2da0d6d34b35fdf/users/1e81eef05a4a40cbaab2a3b975f8d583/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:56.147609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68636ede-ab03-4936-b495-cb3641a38496 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.168983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 790/1579] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:56.170093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa520-c9db-4bed-91a7-cd763679ecb3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.210763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68636ede-ab03-4936-b495-cb3641a38496 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.233559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa520-c9db-4bed-91a7-cd763679ecb3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.238688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff265c81-f999-49b1-a0f3-e57035de0cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.239335 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff265c81-f999-49b1-a0f3-e57035de0cfe admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.239611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff265c81-f999-49b1-a0f3-e57035de0cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.239920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff265c81-f999-49b1-a0f3-e57035de0cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.240529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff265c81-f999-49b1-a0f3-e57035de0cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.242414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff265c81-f999-49b1-a0f3-e57035de0cfe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.267279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d274340-fcbe-458c-8255-66577beb91c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.267812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d274340-fcbe-458c-8255-66577beb91c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.268090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d274340-fcbe-458c-8255-66577beb91c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.268380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d274340-fcbe-458c-8255-66577beb91c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.269001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d274340-fcbe-458c-8255-66577beb91c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.270250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 791/1580] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:56.270590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d274340-fcbe-458c-8255-66577beb91c5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.271072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff265c81-f999-49b1-a0f3-e57035de0cfe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.285301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 790/1581] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.285301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d274340-fcbe-458c-8255-66577beb91c5 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.292128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d274340-fcbe-458c-8255-66577beb91c5 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.336026 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff265c81-f999-49b1-a0f3-e57035de0cfe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.340635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fc0e3-889d-4b93-be56-e5064d4aca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.341235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fc0e3-889d-4b93-be56-e5064d4aca86 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.341509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fc0e3-889d-4b93-be56-e5064d4aca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.341778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fc0e3-889d-4b93-be56-e5064d4aca86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.342386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fc0e3-889d-4b93-be56-e5064d4aca86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.344286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fc0e3-889d-4b93-be56-e5064d4aca86 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.347450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cf2b5-9a22-4e2a-aee0-9afc21d049ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.348044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cf2b5-9a22-4e2a-aee0-9afc21d049ee admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.348319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cf2b5-9a22-4e2a-aee0-9afc21d049ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.348592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cf2b5-9a22-4e2a-aee0-9afc21d049ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.349215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cf2b5-9a22-4e2a-aee0-9afc21d049ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.350768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cf2b5-9a22-4e2a-aee0-9afc21d049ee admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.355725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:56.362208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 791/1582] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.363313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cf2b5-9a22-4e2a-aee0-9afc21d049ee admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.369354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:56.374716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cf2b5-9a22-4e2a-aee0-9afc21d049ee admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.381820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 792/1583] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/projects => generated 403 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:56.381979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef38122-2f20-441b-a53c-ee160703c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.382106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f464ad4-91db-4165-a067-dbf3d0b12157 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.382207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38122-2f20-441b-a53c-ee160703c767 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.382318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f464ad4-91db-4165-a067-dbf3d0b12157 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.382416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38122-2f20-441b-a53c-ee160703c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.382511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f464ad4-91db-4165-a067-dbf3d0b12157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.382611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38122-2f20-441b-a53c-ee160703c767 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.382611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef38122-2f20-441b-a53c-ee160703c767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.383381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef38122-2f20-441b-a53c-ee160703c767 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.399830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9f464ad4-91db-4165-a067-dbf3d0b12157 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:56.401142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f464ad4-91db-4165-a067-dbf3d0b12157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:56.408694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 792/1584] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/users => generated 302 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:56.408694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6312d10-64b6-485a-8078-245fc7d31efe None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.408694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6312d10-64b6-485a-8078-245fc7d31efe None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.408694 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6312d10-64b6-485a-8078-245fc7d31efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.435823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c6312d10-64b6-485a-8078-245fc7d31efe None None] MFA Rules not processed for user `1e81eef05a4a40cbaab2a3b975f8d583`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:56.437553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c6312d10-64b6-485a-8078-245fc7d31efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:56.465441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 793/1585] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:56.466601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f464ad4-91db-4165-a067-dbf3d0b12157 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.476068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f464ad4-91db-4165-a067-dbf3d0b12157 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.480684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5f1ec-d302-4a95-8ba6-618dbd547ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.481333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5f1ec-d302-4a95-8ba6-618dbd547ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.481551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5f1ec-d302-4a95-8ba6-618dbd547ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.481817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5f1ec-d302-4a95-8ba6-618dbd547ebd admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.482432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5f1ec-d302-4a95-8ba6-618dbd547ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.484900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5f1ec-d302-4a95-8ba6-618dbd547ebd admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.505541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 793/1586] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:56.506419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6312d10-64b6-485a-8078-245fc7d31efe None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.509944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 794/1587] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/users => generated 283 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:56.509944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5f1ec-d302-4a95-8ba6-618dbd547ebd admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.520174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5f1ec-d302-4a95-8ba6-618dbd547ebd admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.520363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6312d10-64b6-485a-8078-245fc7d31efe None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.525078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a1dc1-86a3-44c4-bd9e-9743b742f9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.525731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c217-bf04-4938-84f2-f2fe9344ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.525848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a1dc1-86a3-44c4-bd9e-9743b742f9ed admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.525941 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c217-bf04-4938-84f2-f2fe9344ced5 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.526048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a1dc1-86a3-44c4-bd9e-9743b742f9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.526142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c217-bf04-4938-84f2-f2fe9344ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.526226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c217-bf04-4938-84f2-f2fe9344ced5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.526306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a1dc1-86a3-44c4-bd9e-9743b742f9ed admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.526852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a1dc1-86a3-44c4-bd9e-9743b742f9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.527002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40c217-bf04-4938-84f2-f2fe9344ced5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.528516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40c217-bf04-4938-84f2-f2fe9344ced5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.529292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a1dc1-86a3-44c4-bd9e-9743b742f9ed admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.542005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 795/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.542972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c217-bf04-4938-84f2-f2fe9344ced5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.556599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c217-bf04-4938-84f2-f2fe9344ced5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.559889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 794/1589] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/users => generated 296 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:56.560653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a1dc1-86a3-44c4-bd9e-9743b742f9ed admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.564000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ab8c4-c82c-4908-b2dd-548d6da7e6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.564414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ab8c4-c82c-4908-b2dd-548d6da7e6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.564689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ab8c4-c82c-4908-b2dd-548d6da7e6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.564962 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ab8c4-c82c-4908-b2dd-548d6da7e6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.565675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ab8c4-c82c-4908-b2dd-548d6da7e6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.567298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ab8c4-c82c-4908-b2dd-548d6da7e6f7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.570931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a1dc1-86a3-44c4-bd9e-9743b742f9ed admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.575859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0cf31-36f3-4189-9a7b-fa477ea3244a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.576520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0cf31-36f3-4189-9a7b-fa477ea3244a admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.576826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0cf31-36f3-4189-9a7b-fa477ea3244a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.577102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0cf31-36f3-4189-9a7b-fa477ea3244a admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.577742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0cf31-36f3-4189-9a7b-fa477ea3244a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.581389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 796/1590] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.581389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ab8c4-c82c-4908-b2dd-548d6da7e6f7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.581678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0cf31-36f3-4189-9a7b-fa477ea3244a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.588267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ab8c4-c82c-4908-b2dd-548d6da7e6f7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.593510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fbc5c-c8c1-4243-8fa5-eed609e54623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.594072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fbc5c-c8c1-4243-8fa5-eed609e54623 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.594364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fbc5c-c8c1-4243-8fa5-eed609e54623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.594596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fbc5c-c8c1-4243-8fa5-eed609e54623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.595262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fbc5c-c8c1-4243-8fa5-eed609e54623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.596855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fbc5c-c8c1-4243-8fa5-eed609e54623 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.601688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 795/1591] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/users => generated 302 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:56.602719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0cf31-36f3-4189-9a7b-fa477ea3244a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.607072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 797/1592] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.608027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fbc5c-c8c1-4243-8fa5-eed609e54623 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.618174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fbc5c-c8c1-4243-8fa5-eed609e54623 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.622681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a4a2b-6213-4158-bf16-c25c2ac36966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.623280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a4a2b-6213-4158-bf16-c25c2ac36966 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.623554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a4a2b-6213-4158-bf16-c25c2ac36966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.623930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a4a2b-6213-4158-bf16-c25c2ac36966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.624933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a4a2b-6213-4158-bf16-c25c2ac36966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.627878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a4a2b-6213-4158-bf16-c25c2ac36966 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.640780 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 798/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.641552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a4a2b-6213-4158-bf16-c25c2ac36966 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.651014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a4a2b-6213-4158-bf16-c25c2ac36966 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.656171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b14fce-5c4e-4c40-8896-b7d4400f3b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.657022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b14fce-5c4e-4c40-8896-b7d4400f3b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.657366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b14fce-5c4e-4c40-8896-b7d4400f3b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.657708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b14fce-5c4e-4c40-8896-b7d4400f3b6a admin admin] PATH_INFO: `/v3/projects/66439eb22e4a4938b0522bfd356ca3f9/users/2dd3653b931643318f1a1b0820620277/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.666419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0cf31-36f3-4189-9a7b-fa477ea3244a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.670071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc2211-b591-4f89-9f0c-cb413c73a5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.671307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc2211-b591-4f89-9f0c-cb413c73a5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.671578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc2211-b591-4f89-9f0c-cb413c73a5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.671904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc2211-b591-4f89-9f0c-cb413c73a5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.672531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b14fce-5c4e-4c40-8896-b7d4400f3b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=66439eb22e4a4938b0522bfd356ca3f9, user_id=2dd3653b931643318f1a1b0820620277, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.675186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b14fce-5c4e-4c40-8896-b7d4400f3b6a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.680838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc2211-b591-4f89-9f0c-cb413c73a5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.682250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc2211-b591-4f89-9f0c-cb413c73a5c1 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.687732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 799/1594] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:56 2022] PUT /identity/v3/projects/66439eb22e4a4938b0522bfd356ca3f9/users/2dd3653b931643318f1a1b0820620277/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:56.688925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172346a8-3a0d-4f09-bdc0-7e226d6540eb None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.689201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172346a8-3a0d-4f09-bdc0-7e226d6540eb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.689468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172346a8-3a0d-4f09-bdc0-7e226d6540eb None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.690682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 800/1595] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 10 10:02:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:02:56.695532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 796/1596] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:02:56.695671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172346a8-3a0d-4f09-bdc0-7e226d6540eb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.696375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc2211-b591-4f89-9f0c-cb413c73a5c1 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.705595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc2211-b591-4f89-9f0c-cb413c73a5c1 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.751137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:56.773796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8333d-3c2f-4dc1-946b-5c11be798aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.774043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172346a8-3a0d-4f09-bdc0-7e226d6540eb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.774386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8333d-3c2f-4dc1-946b-5c11be798aab admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.774661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8333d-3c2f-4dc1-946b-5c11be798aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.774930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8333d-3c2f-4dc1-946b-5c11be798aab admin admin] PATH_INFO: `/v3/projects/90bcc8b088e34fb98186f4b1faa47983/users/60046731fbb14ee1897a2c4fddf52b92/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.780977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204f150-24a8-4b2e-97da-eb10fccb07ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.781752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204f150-24a8-4b2e-97da-eb10fccb07ea admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.782158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204f150-24a8-4b2e-97da-eb10fccb07ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.782532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204f150-24a8-4b2e-97da-eb10fccb07ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.783713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204f150-24a8-4b2e-97da-eb10fccb07ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.785549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204f150-24a8-4b2e-97da-eb10fccb07ea admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.792276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8333d-3c2f-4dc1-946b-5c11be798aab admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bcc8b088e34fb98186f4b1faa47983, user_id=60046731fbb14ee1897a2c4fddf52b92, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.796121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8333d-3c2f-4dc1-946b-5c11be798aab admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.797861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 801/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.801597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204f150-24a8-4b2e-97da-eb10fccb07ea admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.812385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 797/1598] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:56 2022] PUT /identity/v3/projects/90bcc8b088e34fb98186f4b1faa47983/users/60046731fbb14ee1897a2c4fddf52b92/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:56.813468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204f150-24a8-4b2e-97da-eb10fccb07ea admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.813896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8333d-3c2f-4dc1-946b-5c11be798aab admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.830682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:56.890144 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc848c-d7c6-46d9-b65d-fbaef7f1c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.891073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc848c-d7c6-46d9-b65d-fbaef7f1c5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.891466 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc848c-d7c6-46d9-b65d-fbaef7f1c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.892137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc848c-d7c6-46d9-b65d-fbaef7f1c5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.892802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc848c-d7c6-46d9-b65d-fbaef7f1c5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.894441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc848c-d7c6-46d9-b65d-fbaef7f1c5eb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.895273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8333d-3c2f-4dc1-946b-5c11be798aab admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.900467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d0b4b-db9b-43d9-b0af-1661a5a227e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.901171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d0b4b-db9b-43d9-b0af-1661a5a227e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.901512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d0b4b-db9b-43d9-b0af-1661a5a227e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.901914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d0b4b-db9b-43d9-b0af-1661a5a227e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.903120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d0b4b-db9b-43d9-b0af-1661a5a227e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.909082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d0b4b-db9b-43d9-b0af-1661a5a227e9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.909273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 802/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.909273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc848c-d7c6-46d9-b65d-fbaef7f1c5eb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.922672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 798/1600] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.922672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b08d76-a76c-4fd5-99fb-aa3a504404be None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.922672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b08d76-a76c-4fd5-99fb-aa3a504404be None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.922672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b08d76-a76c-4fd5-99fb-aa3a504404be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.923328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc848c-d7c6-46d9-b65d-fbaef7f1c5eb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.929146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93102d3-9fb2-48cf-9e99-cc32fa824d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.929887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93102d3-9fb2-48cf-9e99-cc32fa824d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.930469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93102d3-9fb2-48cf-9e99-cc32fa824d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.930892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93102d3-9fb2-48cf-9e99-cc32fa824d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.931769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93102d3-9fb2-48cf-9e99-cc32fa824d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.933713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93102d3-9fb2-48cf-9e99-cc32fa824d40 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.944559 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-15b08d76-a76c-4fd5-99fb-aa3a504404be None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:56.945825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-15b08d76-a76c-4fd5-99fb-aa3a504404be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:56.948442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 803/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:56.953210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdd943-fbe2-4e08-bb2e-29882299b63e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.953501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdd943-fbe2-4e08-bb2e-29882299b63e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.953807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdd943-fbe2-4e08-bb2e-29882299b63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.956168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 799/1602] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:56.960803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b08d76-a76c-4fd5-99fb-aa3a504404be None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:56.977045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b08d76-a76c-4fd5-99fb-aa3a504404be None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:56.984152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082043f-6a25-4a1d-978e-001e093f3caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:56.984867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082043f-6a25-4a1d-978e-001e093f3caf admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:56.985143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082043f-6a25-4a1d-978e-001e093f3caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:56.985968 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082043f-6a25-4a1d-978e-001e093f3caf admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:56.986377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082043f-6a25-4a1d-978e-001e093f3caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:56.988147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082043f-6a25-4a1d-978e-001e093f3caf admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:56.999813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 800/1603] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:57.001427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a9f26-3212-4c22-852c-0b478196b939 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.001785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a9f26-3212-4c22-852c-0b478196b939 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.002017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a9f26-3212-4c22-852c-0b478196b939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.003345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8ecdd943-fbe2-4e08-bb2e-29882299b63e None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:57.004546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8ecdd943-fbe2-4e08-bb2e-29882299b63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:57.032854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-de1a9f26-3212-4c22-852c-0b478196b939 None None] MFA Rules not processed for user `60046731fbb14ee1897a2c4fddf52b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:57.034408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-de1a9f26-3212-4c22-852c-0b478196b939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:57.080388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 804/1604] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:57.081652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdd943-fbe2-4e08-bb2e-29882299b63e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:57.092056 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdd943-fbe2-4e08-bb2e-29882299b63e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:57.096745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c3e32-424f-4ea4-9172-5de1fbc8589b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:57.097340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c3e32-424f-4ea4-9172-5de1fbc8589b admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.097618 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c3e32-424f-4ea4-9172-5de1fbc8589b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.097911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c3e32-424f-4ea4-9172-5de1fbc8589b admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.098527 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c3e32-424f-4ea4-9172-5de1fbc8589b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:57.100364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c3e32-424f-4ea4-9172-5de1fbc8589b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:57.115797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 805/1605] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:57.115797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c3e32-424f-4ea4-9172-5de1fbc8589b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:57.116079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 801/1606] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:57.116079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a9f26-3212-4c22-852c-0b478196b939 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:57.130617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a9f26-3212-4c22-852c-0b478196b939 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:57.131246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c3e32-424f-4ea4-9172-5de1fbc8589b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:57.136832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9516d-7e15-4dea-a5f4-ed84af096909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:57.137323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477d6d8-e462-4436-8fde-5fa1dc0dbd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:57.137627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9516d-7e15-4dea-a5f4-ed84af096909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.137921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9516d-7e15-4dea-a5f4-ed84af096909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.138199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9516d-7e15-4dea-a5f4-ed84af096909 admin admin] PATH_INFO: `/v3/projects/95cda9ccaaea4bae980539acb2b5669d/users/e9c60abd394946dbacfe98a5103db5b2/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.139044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477d6d8-e462-4436-8fde-5fa1dc0dbd86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.140085 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477d6d8-e462-4436-8fde-5fa1dc0dbd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.140604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477d6d8-e462-4436-8fde-5fa1dc0dbd86 admin admin] PATH_INFO: `/v3/projects/b6f12c730fb242e9868838c412572e0e/users/ae0020c2bbf54f969b814a4c6fc77bf3/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.158944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477d6d8-e462-4436-8fde-5fa1dc0dbd86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f12c730fb242e9868838c412572e0e, user_id=ae0020c2bbf54f969b814a4c6fc77bf3, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:57.159531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9516d-7e15-4dea-a5f4-ed84af096909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95cda9ccaaea4bae980539acb2b5669d, user_id=e9c60abd394946dbacfe98a5103db5b2, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:57.161819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477d6d8-e462-4436-8fde-5fa1dc0dbd86 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:57.162436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9516d-7e15-4dea-a5f4-ed84af096909 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:57.179510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 806/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:57 2022] PUT /identity/v3/projects/95cda9ccaaea4bae980539acb2b5669d/users/e9c60abd394946dbacfe98a5103db5b2/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:57.180447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9516d-7e15-4dea-a5f4-ed84af096909 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:57.183499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a3c9516d-7e15-4dea-a5f4-ed84af096909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:57.192865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 802/1608] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:57 2022] PUT /identity/v3/projects/b6f12c730fb242e9868838c412572e0e/users/ae0020c2bbf54f969b814a4c6fc77bf3/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:57.193747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477d6d8-e462-4436-8fde-5fa1dc0dbd86 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:57.210287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:57.263861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:57.282243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9516d-7e15-4dea-a5f4-ed84af096909 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:57.301154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477d6d8-e462-4436-8fde-5fa1dc0dbd86 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:57.307024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82e21-f367-455f-ad99-01fb84e141f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:57.307706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82e21-f367-455f-ad99-01fb84e141f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.307979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82e21-f367-455f-ad99-01fb84e141f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.308252 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82e21-f367-455f-ad99-01fb84e141f5 admin admin] PATH_INFO: `/v3/projects/fbbf8047a09b4fee9a12288ef323338c/users/befbca3738f0481296b63c70d6d2a797/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.323860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d82e21-f367-455f-ad99-01fb84e141f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbbf8047a09b4fee9a12288ef323338c, user_id=befbca3738f0481296b63c70d6d2a797, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:57.326512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d82e21-f367-455f-ad99-01fb84e141f5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:57.344004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 803/1609] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:57 2022] PUT /identity/v3/projects/fbbf8047a09b4fee9a12288ef323338c/users/befbca3738f0481296b63c70d6d2a797/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:57.344897 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82e21-f367-455f-ad99-01fb84e141f5 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:57.349323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ca05e-5520-4901-b91c-443f5b866e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:57.350270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ca05e-5520-4901-b91c-443f5b866e31 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.350548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ca05e-5520-4901-b91c-443f5b866e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.350820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ca05e-5520-4901-b91c-443f5b866e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.360787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ca05e-5520-4901-b91c-443f5b866e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:57.362259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ca05e-5520-4901-b91c-443f5b866e31 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:57.372494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 807/1610] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:02:57.373348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ca05e-5520-4901-b91c-443f5b866e31 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:57.385354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ca05e-5520-4901-b91c-443f5b866e31 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:57.451802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82e21-f367-455f-ad99-01fb84e141f5 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:57.460732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9c55-2d33-4cb8-886a-807b8336a5c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:57.461481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9c55-2d33-4cb8-886a-807b8336a5c5 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.462312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9c55-2d33-4cb8-886a-807b8336a5c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.462682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9c55-2d33-4cb8-886a-807b8336a5c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.481766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4640725-93af-4a9c-863c-e0ed70086cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:57.482700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4640725-93af-4a9c-863c-e0ed70086cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.483419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4640725-93af-4a9c-863c-e0ed70086cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.483869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4640725-93af-4a9c-863c-e0ed70086cc5 admin admin] PATH_INFO: `/v3/projects/fb10b19835d842da81152aab15fae61b/users/32fdacb7de1b46aea9a9b515b0a51fbe/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.508588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4640725-93af-4a9c-863c-e0ed70086cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb10b19835d842da81152aab15fae61b, user_id=32fdacb7de1b46aea9a9b515b0a51fbe, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:57.512897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4640725-93af-4a9c-863c-e0ed70086cc5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:57.542146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 808/1611] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:57 2022] PUT /identity/v3/projects/fb10b19835d842da81152aab15fae61b/users/32fdacb7de1b46aea9a9b515b0a51fbe/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:57.543353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4640725-93af-4a9c-863c-e0ed70086cc5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:57.570279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9c55-2d33-4cb8-886a-807b8336a5c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:57.577821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9c55-2d33-4cb8-886a-807b8336a5c5 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:57.622693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 804/1612] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:02:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:02:57.628009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca8271-60e9-4f8d-85bb-eec4fe104d1f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.628009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca8271-60e9-4f8d-85bb-eec4fe104d1f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.628234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca8271-60e9-4f8d-85bb-eec4fe104d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.709818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6aca8271-60e9-4f8d-85bb-eec4fe104d1f None None] MFA Rules not processed for user `e9c60abd394946dbacfe98a5103db5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:57.711224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6aca8271-60e9-4f8d-85bb-eec4fe104d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:57.711809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4640725-93af-4a9c-863c-e0ed70086cc5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:57.723102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd02fd-af26-40bf-995f-f66fe54ee3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:57.724544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd02fd-af26-40bf-995f-f66fe54ee3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.724939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd02fd-af26-40bf-995f-f66fe54ee3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.726263 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd02fd-af26-40bf-995f-f66fe54ee3f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.726868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd02fd-af26-40bf-995f-f66fe54ee3f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:57.735763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd02fd-af26-40bf-995f-f66fe54ee3f7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:57.747330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:57.793236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 809/1613] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:57 2022] POST /identity/v3/projects => generated 357 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:57.794741 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec38f0-c4e9-4549-bde7-cac1e6a7b179 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.795332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec38f0-c4e9-4549-bde7-cac1e6a7b179 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.797291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec38f0-c4e9-4549-bde7-cac1e6a7b179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.832901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:57.865295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-02ec38f0-c4e9-4549-bde7-cac1e6a7b179 None None] MFA Rules not processed for user `ae0020c2bbf54f969b814a4c6fc77bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:57.870560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-02ec38f0-c4e9-4549-bde7-cac1e6a7b179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:57.870713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 805/1614] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:57.871432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55bce4-283c-4552-aaef-101a97d8fa49 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:57.872465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55bce4-283c-4552-aaef-101a97d8fa49 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:57.874510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55bce4-283c-4552-aaef-101a97d8fa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:57.922852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-bd55bce4-283c-4552-aaef-101a97d8fa49 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:57.927556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bd55bce4-283c-4552-aaef-101a97d8fa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:58.034285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 810/1615] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:58.046842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 806/1616] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:58.046842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55bce4-283c-4552-aaef-101a97d8fa49 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.051487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06220365-5246-4274-86c6-b4d65ff4e20e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.052094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06220365-5246-4274-86c6-b4d65ff4e20e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.053538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06220365-5246-4274-86c6-b4d65ff4e20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.059251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55bce4-283c-4552-aaef-101a97d8fa49 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.067375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a14d4-3282-4072-93c0-d2ac4d73e50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.068595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a14d4-3282-4072-93c0-d2ac4d73e50e admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.068988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a14d4-3282-4072-93c0-d2ac4d73e50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.069367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a14d4-3282-4072-93c0-d2ac4d73e50e admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.071261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a14d4-3282-4072-93c0-d2ac4d73e50e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.073911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a14d4-3282-4072-93c0-d2ac4d73e50e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.108562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-06220365-5246-4274-86c6-b4d65ff4e20e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:58.110591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 807/1617] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:58 2022] POST /identity/v3/users => generated 285 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:58.112607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a14d4-3282-4072-93c0-d2ac4d73e50e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.115480 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-06220365-5246-4274-86c6-b4d65ff4e20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:58.135661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a14d4-3282-4072-93c0-d2ac4d73e50e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.144409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 811/1618] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:58.144409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06220365-5246-4274-86c6-b4d65ff4e20e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.145692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8118c73-d94a-4bd8-955a-06befd2aa4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.145878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8118c73-d94a-4bd8-955a-06befd2aa4bd admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.146032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8118c73-d94a-4bd8-955a-06befd2aa4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.146698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8118c73-d94a-4bd8-955a-06befd2aa4bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.146996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8118c73-d94a-4bd8-955a-06befd2aa4bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.149281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8118c73-d94a-4bd8-955a-06befd2aa4bd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.185885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 808/1619] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:02:58 2022] POST /identity/v3/projects => generated 403 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:58.187083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8118c73-d94a-4bd8-955a-06befd2aa4bd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.292287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06220365-5246-4274-86c6-b4d65ff4e20e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.301093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8118c73-d94a-4bd8-955a-06befd2aa4bd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.315733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdc3c4-78d5-42ca-8bf4-acc49e4305dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.316428 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdc3c4-78d5-42ca-8bf4-acc49e4305dc service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.316654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdc3c4-78d5-42ca-8bf4-acc49e4305dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.316988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdc3c4-78d5-42ca-8bf4-acc49e4305dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.384454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:58.439099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdc3c4-78d5-42ca-8bf4-acc49e4305dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.439756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178116a6-ae67-4d4f-93c6-a0ac0749df39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.440397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178116a6-ae67-4d4f-93c6-a0ac0749df39 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.440650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdc3c4-78d5-42ca-8bf4-acc49e4305dc service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.440792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178116a6-ae67-4d4f-93c6-a0ac0749df39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.443180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178116a6-ae67-4d4f-93c6-a0ac0749df39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.453500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 809/1620] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:02:58.454362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdc3c4-78d5-42ca-8bf4-acc49e4305dc service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.465302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdc3c4-78d5-42ca-8bf4-acc49e4305dc service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.471404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2ae58-b1ca-4011-bbf9-d1eb0bd27eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.472169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2ae58-b1ca-4011-bbf9-d1eb0bd27eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.472454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2ae58-b1ca-4011-bbf9-d1eb0bd27eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.472726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2ae58-b1ca-4011-bbf9-d1eb0bd27eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.473369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2ae58-b1ca-4011-bbf9-d1eb0bd27eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.475150 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2ae58-b1ca-4011-bbf9-d1eb0bd27eaa admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.493071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 810/1621] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:58.493997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2ae58-b1ca-4011-bbf9-d1eb0bd27eaa admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.506060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2ae58-b1ca-4011-bbf9-d1eb0bd27eaa admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.510704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f7e2b-35c6-4a9a-ab2d-7f8af3849354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.511510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f7e2b-35c6-4a9a-ab2d-7f8af3849354 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.511787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f7e2b-35c6-4a9a-ab2d-7f8af3849354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.512069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f7e2b-35c6-4a9a-ab2d-7f8af3849354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.527495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f7e2b-35c6-4a9a-ab2d-7f8af3849354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.531164 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f7e2b-35c6-4a9a-ab2d-7f8af3849354 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.566774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 811/1622] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:02:58.568151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f7e2b-35c6-4a9a-ab2d-7f8af3849354 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.584526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f7e2b-35c6-4a9a-ab2d-7f8af3849354 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.596076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e918ab-a730-4d90-a5e9-502ba88aaefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.597043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e918ab-a730-4d90-a5e9-502ba88aaefe admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.597474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e918ab-a730-4d90-a5e9-502ba88aaefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.597854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e918ab-a730-4d90-a5e9-502ba88aaefe admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.598916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178116a6-ae67-4d4f-93c6-a0ac0749df39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.600050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e918ab-a730-4d90-a5e9-502ba88aaefe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.600405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178116a6-ae67-4d4f-93c6-a0ac0749df39 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.603410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e918ab-a730-4d90-a5e9-502ba88aaefe admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.612947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:58.620826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 812/1623] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:02:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:02:58.622279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178116a6-ae67-4d4f-93c6-a0ac0749df39 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.640477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 812/1624] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:02:58 2022] POST /identity/v3/users => generated 308 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:02:58.642208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178116a6-ae67-4d4f-93c6-a0ac0749df39 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.644281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e918ab-a730-4d90-a5e9-502ba88aaefe admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.654035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b59b0e-6af5-4832-98cb-744b32e50a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.654892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b59b0e-6af5-4832-98cb-744b32e50a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.655732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b59b0e-6af5-4832-98cb-744b32e50a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.656096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b59b0e-6af5-4832-98cb-744b32e50a53 admin admin] PATH_INFO: `/v3/projects/180353c3abc04d35b6c6b3e78be90979/users/ffb17d1b3f444be78a8acc8bd151d101/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.662990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e918ab-a730-4d90-a5e9-502ba88aaefe admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.685129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883192c-1758-436c-a8d0-7adf3f4374dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.685129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883192c-1758-436c-a8d0-7adf3f4374dd admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.685129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883192c-1758-436c-a8d0-7adf3f4374dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.685129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883192c-1758-436c-a8d0-7adf3f4374dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.685481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883192c-1758-436c-a8d0-7adf3f4374dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.685481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883192c-1758-436c-a8d0-7adf3f4374dd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.696826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b59b0e-6af5-4832-98cb-744b32e50a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=180353c3abc04d35b6c6b3e78be90979, user_id=ffb17d1b3f444be78a8acc8bd151d101, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.706885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b59b0e-6af5-4832-98cb-744b32e50a53 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.727228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 813/1625] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:02:58.733749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883192c-1758-436c-a8d0-7adf3f4374dd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.742594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 813/1626] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:58 2022] PUT /identity/v3/projects/180353c3abc04d35b6c6b3e78be90979/users/ffb17d1b3f444be78a8acc8bd151d101/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:58.754192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b59b0e-6af5-4832-98cb-744b32e50a53 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.773423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883192c-1758-436c-a8d0-7adf3f4374dd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.906175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b59b0e-6af5-4832-98cb-744b32e50a53 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.912426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e690be-7cf5-4724-a981-1d94fd8c9857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.913051 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e690be-7cf5-4724-a981-1d94fd8c9857 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.913332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e690be-7cf5-4724-a981-1d94fd8c9857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.913608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e690be-7cf5-4724-a981-1d94fd8c9857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.914260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e690be-7cf5-4724-a981-1d94fd8c9857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.915896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e690be-7cf5-4724-a981-1d94fd8c9857 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.919462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b4e93-134b-4b1a-b6d1-3f22fcabba8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.926724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b4e93-134b-4b1a-b6d1-3f22fcabba8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.927370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b4e93-134b-4b1a-b6d1-3f22fcabba8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.927370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b4e93-134b-4b1a-b6d1-3f22fcabba8c admin admin] PATH_INFO: `/v3/projects/dd3a2092704a491cbfa3ba054f1f8f49/users/685ee4583a0b45449a23c2769780c012/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.932498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 814/1627] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:58.934181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b3bd9-c4bd-40c0-801b-8219fcd538e5 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.934625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b3bd9-c4bd-40c0-801b-8219fcd538e5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.934976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b3bd9-c4bd-40c0-801b-8219fcd538e5 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.937461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 815/1628] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 10 10:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:02:58.938280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b3bd9-c4bd-40c0-801b-8219fcd538e5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:58.959369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b3bd9-c4bd-40c0-801b-8219fcd538e5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:58.970843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947161e-caf2-42f0-bde8-b0e075b1c6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:58.971833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947161e-caf2-42f0-bde8-b0e075b1c6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:58.972228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947161e-caf2-42f0-bde8-b0e075b1c6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:58.972577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947161e-caf2-42f0-bde8-b0e075b1c6fc admin admin] PATH_INFO: `/v3/projects/180353c3abc04d35b6c6b3e78be90979/users/ffb17d1b3f444be78a8acc8bd151d101/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:58.977682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947161e-caf2-42f0-bde8-b0e075b1c6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=180353c3abc04d35b6c6b3e78be90979, user_id=ffb17d1b3f444be78a8acc8bd151d101, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.980619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947161e-caf2-42f0-bde8-b0e075b1c6fc admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:58.989656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b4e93-134b-4b1a-b6d1-3f22fcabba8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd3a2092704a491cbfa3ba054f1f8f49, user_id=685ee4583a0b45449a23c2769780c012, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:58.994071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b4e93-134b-4b1a-b6d1-3f22fcabba8c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:59.019446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 816/1629] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:58 2022] PUT /identity/v3/projects/180353c3abc04d35b6c6b3e78be90979/users/ffb17d1b3f444be78a8acc8bd151d101/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:59.021309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94ea65-b0a3-471e-a884-664e0313ea28 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:59.021674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94ea65-b0a3-471e-a884-664e0313ea28 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:59.022041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94ea65-b0a3-471e-a884-664e0313ea28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:59.027105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 814/1630] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:58 2022] PUT /identity/v3/projects/dd3a2092704a491cbfa3ba054f1f8f49/users/685ee4583a0b45449a23c2769780c012/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:02:59.029834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fff573-146e-4c0f-9a06-5919c91b9af3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:59.030267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fff573-146e-4c0f-9a06-5919c91b9af3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:59.030685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fff573-146e-4c0f-9a06-5919c91b9af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:59.057495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8e94ea65-b0a3-471e-a884-664e0313ea28 None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:59.057495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8e94ea65-b0a3-471e-a884-664e0313ea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:59.089612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-03fff573-146e-4c0f-9a06-5919c91b9af3 None None] MFA Rules not processed for user `ffb17d1b3f444be78a8acc8bd151d101`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:59.091047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-03fff573-146e-4c0f-9a06-5919c91b9af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:59.196313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:59.211762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 817/1631] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 10 10:02:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:59.212957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94ea65-b0a3-471e-a884-664e0313ea28 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:59.254395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 815/1632] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:59.261040 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0a796-25b4-4602-a367-d4e4774dbac0 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:59.261040 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0a796-25b4-4602-a367-d4e4774dbac0 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:59.262997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0a796-25b4-4602-a367-d4e4774dbac0 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:59.266299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 816/1633] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 10 10:02:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:02:59.270671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f64a22-ef66-4606-a835-813c4c322f8e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:59.270984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f64a22-ef66-4606-a835-813c4c322f8e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:59.271256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f64a22-ef66-4606-a835-813c4c322f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:59.314700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-01f64a22-ef66-4606-a835-813c4c322f8e None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:59.317261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-01f64a22-ef66-4606-a835-813c4c322f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:59.336347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:02:59.424216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 817/1634] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:02:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:59.424216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f64a22-ef66-4606-a835-813c4c322f8e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:59.428395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94ea65-b0a3-471e-a884-664e0313ea28 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:59.428395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131362ac-24a6-40c6-96d3-94ba12cab2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:59.428395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131362ac-24a6-40c6-96d3-94ba12cab2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:59.428395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131362ac-24a6-40c6-96d3-94ba12cab2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:59.432700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131362ac-24a6-40c6-96d3-94ba12cab2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:59.432700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131362ac-24a6-40c6-96d3-94ba12cab2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:59.432700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131362ac-24a6-40c6-96d3-94ba12cab2e5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:59.462279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 818/1635] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:02:59.463831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131362ac-24a6-40c6-96d3-94ba12cab2e5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:59.485502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131362ac-24a6-40c6-96d3-94ba12cab2e5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:59.504252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6a48f-1298-4ddb-af12-d01a2de0f6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:59.506329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6a48f-1298-4ddb-af12-d01a2de0f6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:59.507974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6a48f-1298-4ddb-af12-d01a2de0f6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:59.508588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6a48f-1298-4ddb-af12-d01a2de0f6dc admin admin] PATH_INFO: `/v3/projects/dd3a2092704a491cbfa3ba054f1f8f49/users/685ee4583a0b45449a23c2769780c012/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:59.512927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6a48f-1298-4ddb-af12-d01a2de0f6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd3a2092704a491cbfa3ba054f1f8f49, user_id=685ee4583a0b45449a23c2769780c012, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:59.522242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6a48f-1298-4ddb-af12-d01a2de0f6dc admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:59.531713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f64a22-ef66-4606-a835-813c4c322f8e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:59.536245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e40d2-6ead-45ce-8026-bb7357567c8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:59.537236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e40d2-6ead-45ce-8026-bb7357567c8b service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:59.537525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e40d2-6ead-45ce-8026-bb7357567c8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:59.537861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e40d2-6ead-45ce-8026-bb7357567c8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:59.558287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e40d2-6ead-45ce-8026-bb7357567c8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:59.559210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 819/1636] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:02:59 2022] PUT /identity/v3/projects/dd3a2092704a491cbfa3ba054f1f8f49/users/685ee4583a0b45449a23c2769780c012/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:02:59.560982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbb476-7c88-4b3f-9ac8-87b779078ca7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:59.561533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e40d2-6ead-45ce-8026-bb7357567c8b service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:59.562131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbb476-7c88-4b3f-9ac8-87b779078ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:59.562705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbb476-7c88-4b3f-9ac8-87b779078ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:59.587767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 818/1637] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:02:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:02:59.634537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6fbbb476-7c88-4b3f-9ac8-87b779078ca7 None None] MFA Rules not processed for user `685ee4583a0b45449a23c2769780c012`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:02:59.636572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6fbbb476-7c88-4b3f-9ac8-87b779078ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:02:59.670449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e40d2-6ead-45ce-8026-bb7357567c8b service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:02:59.681971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e40d2-6ead-45ce-8026-bb7357567c8b service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:02:59.791186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471260a-4d30-4cac-b7de-8b7fe0deeb1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:02:59.791993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471260a-4d30-4cac-b7de-8b7fe0deeb1b service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:02:59.793768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471260a-4d30-4cac-b7de-8b7fe0deeb1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:02:59.794222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471260a-4d30-4cac-b7de-8b7fe0deeb1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:02:59.840377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 820/1638] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:02:59 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:02:59.942065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471260a-4d30-4cac-b7de-8b7fe0deeb1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:02:59.943751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471260a-4d30-4cac-b7de-8b7fe0deeb1b service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:02:59.972233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 819/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:02:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:00.036555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbb476-7c88-4b3f-9ac8-87b779078ca7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:00.048587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbb476-7c88-4b3f-9ac8-87b779078ca7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:00.064130 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18e1a5-22d7-477e-b3ef-0054f7a507e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:00.064829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18e1a5-22d7-477e-b3ef-0054f7a507e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:00.066186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18e1a5-22d7-477e-b3ef-0054f7a507e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:00.067108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18e1a5-22d7-477e-b3ef-0054f7a507e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:00.089376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18e1a5-22d7-477e-b3ef-0054f7a507e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:00.096247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18e1a5-22d7-477e-b3ef-0054f7a507e9 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:00.153411 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 821/1640] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:00 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:00.156933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0341898-4dbc-47f7-b37d-75e61841c52e None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:00.157601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0341898-4dbc-47f7-b37d-75e61841c52e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:00.157971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0341898-4dbc-47f7-b37d-75e61841c52e None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:00.160226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 820/1641] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 10 10:03:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:03:00.161523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18e1a5-22d7-477e-b3ef-0054f7a507e9 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:00.172905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d54eb-b1cb-464f-a92f-fbe9324311c1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:00.173498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d54eb-b1cb-464f-a92f-fbe9324311c1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:00.173995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d54eb-b1cb-464f-a92f-fbe9324311c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:00.177878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18e1a5-22d7-477e-b3ef-0054f7a507e9 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:00.196748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46b3f3-9dc9-42e8-9e74-ebc2fa96c926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:00.197994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46b3f3-9dc9-42e8-9e74-ebc2fa96c926 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:00.198496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46b3f3-9dc9-42e8-9e74-ebc2fa96c926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:00.199032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46b3f3-9dc9-42e8-9e74-ebc2fa96c926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:00.219220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f70d54eb-b1cb-464f-a92f-fbe9324311c1 None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:00.222760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f70d54eb-b1cb-464f-a92f-fbe9324311c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:00.245483 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 821/1642] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:03:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:00.307062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d54eb-b1cb-464f-a92f-fbe9324311c1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:00.323875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d54eb-b1cb-464f-a92f-fbe9324311c1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:00.332602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e62bbd-6485-49af-869e-e53ac33659ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:00.333961 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e62bbd-6485-49af-869e-e53ac33659ed service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:00.334823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e62bbd-6485-49af-869e-e53ac33659ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:00.336904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e62bbd-6485-49af-869e-e53ac33659ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:00.354258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46b3f3-9dc9-42e8-9e74-ebc2fa96c926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:00.357687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46b3f3-9dc9-42e8-9e74-ebc2fa96c926 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:00.366728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e62bbd-6485-49af-869e-e53ac33659ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:00.368732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e62bbd-6485-49af-869e-e53ac33659ed service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:00.379837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 822/1643] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:00.384412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:00.418910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 822/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:00.551365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46b3f3-9dc9-42e8-9e74-ebc2fa96c926 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:00.609825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e62bbd-6485-49af-869e-e53ac33659ed service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:00.672673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:00.707401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46b3f3-9dc9-42e8-9e74-ebc2fa96c926 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:00.724861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:00.725150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:00.725438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:00.725714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] PATH_INFO: `/v3/users/1e81eef05a4a40cbaab2a3b975f8d583` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:00.727034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e81eef05a4a40cbaab2a3b975f8d583)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:00.728912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:00.800263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e62bbd-6485-49af-869e-e53ac33659ed service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:00.815217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:00.816261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:00.816699 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:00.817173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] PATH_INFO: `/v3/users/e9c60abd394946dbacfe98a5103db5b2` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:00.821603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9c60abd394946dbacfe98a5103db5b2)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:00.822698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: WARNING py.warnings [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 10:03:00.822698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: warnings.warn(msg) Aug 10 10:03:00.822698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: Aug 10 10:03:00.823833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e81eef05a4a40cbaab2a3b975f8d583'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:00.831887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:00.839622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e81eef05a4a40cbaab2a3b975f8d583'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:00.866378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e81eef05a4a40cbaab2a3b975f8d583'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:00.871712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e81eef05a4a40cbaab2a3b975f8d583'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:00.872084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-74e1cc50-a53f-42d3-94df-f3ef865e651c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e81eef05a4a40cbaab2a3b975f8d583'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:00.897323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 823/1645] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:00 2022] DELETE /identity/v3/users/1e81eef05a4a40cbaab2a3b975f8d583 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:00.901839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049d92a-93d6-4b6e-aecb-9641ee318791 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:00.902161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049d92a-93d6-4b6e-aecb-9641ee318791 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:00.902393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049d92a-93d6-4b6e-aecb-9641ee318791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:00.988148 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3049d92a-93d6-4b6e-aecb-9641ee318791 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:00.989666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3049d92a-93d6-4b6e-aecb-9641ee318791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:01.038594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9c60abd394946dbacfe98a5103db5b2'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:01.055973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9c60abd394946dbacfe98a5103db5b2'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:01.079412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9c60abd394946dbacfe98a5103db5b2'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:01.147578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9c60abd394946dbacfe98a5103db5b2'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:01.150254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9c60abd394946dbacfe98a5103db5b2'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:01.161348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 823/1646] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:00 2022] DELETE /identity/v3/users/e9c60abd394946dbacfe98a5103db5b2 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:01.161602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 824/1647] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:01.170139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a6ee0-cf48-4c45-bd15-d55dacddec30 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:01.170482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a6ee0-cf48-4c45-bd15-d55dacddec30 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:01.170771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a6ee0-cf48-4c45-bd15-d55dacddec30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:01.197005 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:01.199052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:01.200044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-542a6ee0-cf48-4c45-bd15-d55dacddec30 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:01.201349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-542a6ee0-cf48-4c45-bd15-d55dacddec30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:01.289895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 825/1648] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:01.303873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a6ee0-cf48-4c45-bd15-d55dacddec30 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:01.305411 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-542a6ee0-cf48-4c45-bd15-d55dacddec30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:01.318293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:01.366121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9b303-a59d-4895-836a-3e36fe84f163 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:01.376898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d28fe-50ce-471c-a5b5-0c297b36e0c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:01.378350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d28fe-50ce-471c-a5b5-0c297b36e0c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:01.378633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d28fe-50ce-471c-a5b5-0c297b36e0c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:01.378902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d28fe-50ce-471c-a5b5-0c297b36e0c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:01.386024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7e8d28fe-50ce-471c-a5b5-0c297b36e0c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:01.423646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d28fe-50ce-471c-a5b5-0c297b36e0c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:01.425630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d28fe-50ce-471c-a5b5-0c297b36e0c1 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:01.455913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 824/1649] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:01.480462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a6ee0-cf48-4c45-bd15-d55dacddec30 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:01.491246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df660bf-21ac-4231-9fb5-c21f4e488149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:01.492449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df660bf-21ac-4231-9fb5-c21f4e488149 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:01.493026 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df660bf-21ac-4231-9fb5-c21f4e488149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:01.493522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df660bf-21ac-4231-9fb5-c21f4e488149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:01.515963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df660bf-21ac-4231-9fb5-c21f4e488149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:01.524692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df660bf-21ac-4231-9fb5-c21f4e488149 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:01.569475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 826/1650] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:02.668444 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8440f7-dcb2-47b7-b4ee-5029ee469fd1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:02.668912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8440f7-dcb2-47b7-b4ee-5029ee469fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:02.669026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8440f7-dcb2-47b7-b4ee-5029ee469fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:02.783825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-2d8440f7-dcb2-47b7-b4ee-5029ee469fd1 None None] MFA Rules not processed for user `32fdacb7de1b46aea9a9b515b0a51fbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:02.786836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-2d8440f7-dcb2-47b7-b4ee-5029ee469fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:02.805902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:02.965246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 825/1651] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:02 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:02.970450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df660bf-21ac-4231-9fb5-c21f4e488149 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:02.972355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4df660bf-21ac-4231-9fb5-c21f4e488149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:02.997455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df660bf-21ac-4231-9fb5-c21f4e488149 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:03.013744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e36ac-5084-4a0b-b701-a5ffe377b8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:03.015965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e36ac-5084-4a0b-b701-a5ffe377b8da admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:03.016657 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e36ac-5084-4a0b-b701-a5ffe377b8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:03.017188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e36ac-5084-4a0b-b701-a5ffe377b8da admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:03.018300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e36ac-5084-4a0b-b701-a5ffe377b8da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:03.020877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e36ac-5084-4a0b-b701-a5ffe377b8da admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:03.067928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 827/1652] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:02 2022] POST /identity/v3/projects => generated 403 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:03.068291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8440f7-dcb2-47b7-b4ee-5029ee469fd1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:03.096772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8440f7-dcb2-47b7-b4ee-5029ee469fd1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:03.206174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e36ac-5084-4a0b-b701-a5ffe377b8da admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:03.210749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2a7e36ac-5084-4a0b-b701-a5ffe377b8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:03.299148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d981b1-e637-474a-b0a8-1f544be906f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:03.299779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d981b1-e637-474a-b0a8-1f544be906f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:03.300060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d981b1-e637-474a-b0a8-1f544be906f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:03.300332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d981b1-e637-474a-b0a8-1f544be906f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:03.300985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d981b1-e637-474a-b0a8-1f544be906f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:03.302934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d981b1-e637-474a-b0a8-1f544be906f9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:03.347377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 826/1653] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:03 2022] POST /identity/v3/users => generated 308 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:03.351359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d981b1-e637-474a-b0a8-1f544be906f9 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:03.362136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e36ac-5084-4a0b-b701-a5ffe377b8da admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:03.367133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:03.367963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:03.369115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:03.369423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] PATH_INFO: `/v3/projects/99a927c2d1514107b2da0d6d34b35fdf` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:03.375633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a927c2d1514107b2da0d6d34b35fdf)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:03.377773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 10:03:03.377773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Aug 10 10:03:03.377773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: Aug 10 10:03:03.379990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:03.381607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d981b1-e637-474a-b0a8-1f544be906f9 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:03.396497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fb3db-f400-4b35-a3af-328e0caaed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:03.397126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fb3db-f400-4b35-a3af-328e0caaed7e admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:03.397406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fb3db-f400-4b35-a3af-328e0caaed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:03.397682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fb3db-f400-4b35-a3af-328e0caaed7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:03.400876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fb3db-f400-4b35-a3af-328e0caaed7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:03.400876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fb3db-f400-4b35-a3af-328e0caaed7e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:03.433324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 827/1654] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:03:03.438239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fb3db-f400-4b35-a3af-328e0caaed7e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:03.464398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fb3db-f400-4b35-a3af-328e0caaed7e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:03.480864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fd550-9685-49a9-82ea-9f8b50f3e745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:03.486241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fd550-9685-49a9-82ea-9f8b50f3e745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:03.486660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fd550-9685-49a9-82ea-9f8b50f3e745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:03.487087 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fd550-9685-49a9-82ea-9f8b50f3e745 admin admin] PATH_INFO: `/v3/projects/e7261336c06443029fc9c880fc86f80e/users/49e4e90c2c944fabb77bfe1e90d4f4ac/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:03.519463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a927c2d1514107b2da0d6d34b35fdf', 'request_id': 'req-bd814314-4958-4cf2-aa60-58fde3ef93c7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:03.559511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a927c2d1514107b2da0d6d34b35fdf', 'request_id': 'req-bd814314-4958-4cf2-aa60-58fde3ef93c7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:03.571708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fd550-9685-49a9-82ea-9f8b50f3e745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7261336c06443029fc9c880fc86f80e, user_id=49e4e90c2c944fabb77bfe1e90d4f4ac, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:03.575518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fd550-9685-49a9-82ea-9f8b50f3e745 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:03.630697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a927c2d1514107b2da0d6d34b35fdf', 'request_id': 'req-bd814314-4958-4cf2-aa60-58fde3ef93c7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:03.637300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 828/1655] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:03 2022] PUT /identity/v3/projects/e7261336c06443029fc9c880fc86f80e/users/49e4e90c2c944fabb77bfe1e90d4f4ac/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:03.638185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fd550-9685-49a9-82ea-9f8b50f3e745 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:03.648008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-344fd550-9685-49a9-82ea-9f8b50f3e745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:03.677971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] The token cache is being invalidate because project 99a927c2d1514107b2da0d6d34b35fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:03.685693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:03.690190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:03.694869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 828/1656] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:03 2022] DELETE /identity/v3/projects/99a927c2d1514107b2da0d6d34b35fdf => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:03.705560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:03.715676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:03.790097 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:03.897712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fd550-9685-49a9-82ea-9f8b50f3e745 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:03.922738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:03.922738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:03.923306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:03.923593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] PATH_INFO: `/v3/projects/95cda9ccaaea4bae980539acb2b5669d` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:03.926725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] RBAC: Authorizing `identity:delete_project(project_id=95cda9ccaaea4bae980539acb2b5669d)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:03.934151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:03.942136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd814314-4958-4cf2-aa60-58fde3ef93c7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:03.958031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:03.964395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b53b2-3b7b-41dd-8d63-3ce8e80aebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:03.964632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b53b2-3b7b-41dd-8d63-3ce8e80aebee admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:03.964839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b53b2-3b7b-41dd-8d63-3ce8e80aebee admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:03.965120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b53b2-3b7b-41dd-8d63-3ce8e80aebee admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:03.965779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b53b2-3b7b-41dd-8d63-3ce8e80aebee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:03.968514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b53b2-3b7b-41dd-8d63-3ce8e80aebee admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:04.010221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 829/1657] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:04.011636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743accef-0220-44aa-bb39-6178a74ae80f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:04.011921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743accef-0220-44aa-bb39-6178a74ae80f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:04.012295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743accef-0220-44aa-bb39-6178a74ae80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:04.078464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-743accef-0220-44aa-bb39-6178a74ae80f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:04.079774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-743accef-0220-44aa-bb39-6178a74ae80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:04.091157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95cda9ccaaea4bae980539acb2b5669d', 'request_id': 'req-6f99e65d-b913-4d30-8797-ad0c9b0df53d'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.112172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95cda9ccaaea4bae980539acb2b5669d', 'request_id': 'req-6f99e65d-b913-4d30-8797-ad0c9b0df53d'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.131815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95cda9ccaaea4bae980539acb2b5669d', 'request_id': 'req-6f99e65d-b913-4d30-8797-ad0c9b0df53d'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.168269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] The token cache is being invalidate because project 95cda9ccaaea4bae980539acb2b5669d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:04.168746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.170641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.174799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 829/1658] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:03 2022] DELETE /identity/v3/projects/95cda9ccaaea4bae980539acb2b5669d => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:04.181239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:04.185599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:04.337306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 830/1659] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:04.341914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743accef-0220-44aa-bb39-6178a74ae80f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:04.346556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-743accef-0220-44aa-bb39-6178a74ae80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:04.384776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743accef-0220-44aa-bb39-6178a74ae80f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:04.407659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:04.408319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:04.408600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:04.408895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] PATH_INFO: `/v3/users/685ee4583a0b45449a23c2769780c012` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:04.412772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] RBAC: Authorizing `identity:delete_user(user_id=685ee4583a0b45449a23c2769780c012)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:04.414335 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:04.447720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99e65d-b913-4d30-8797-ad0c9b0df53d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:04.462923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4d139-0237-4dd8-a1e9-ce168ce5b6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:04.463708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4d139-0237-4dd8-a1e9-ce168ce5b6cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:04.464567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4d139-0237-4dd8-a1e9-ce168ce5b6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:04.464567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4d139-0237-4dd8-a1e9-ce168ce5b6cb admin admin] PATH_INFO: `/v3/projects/e7261336c06443029fc9c880fc86f80e/users/49e4e90c2c944fabb77bfe1e90d4f4ac/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:04.472763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4d139-0237-4dd8-a1e9-ce168ce5b6cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7261336c06443029fc9c880fc86f80e, user_id=49e4e90c2c944fabb77bfe1e90d4f4ac, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:04.481641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4d139-0237-4dd8-a1e9-ce168ce5b6cb admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:04.521371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 830/1660] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:04 2022] PUT /identity/v3/projects/e7261336c06443029fc9c880fc86f80e/users/49e4e90c2c944fabb77bfe1e90d4f4ac/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:04.521371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93352d-717d-45cf-bbd9-48c4ba78de66 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:04.521371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93352d-717d-45cf-bbd9-48c4ba78de66 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:04.521371 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93352d-717d-45cf-bbd9-48c4ba78de66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:04.556461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685ee4583a0b45449a23c2769780c012'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.571014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685ee4583a0b45449a23c2769780c012'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.608772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ff93352d-717d-45cf-bbd9-48c4ba78de66 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:04.609983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ff93352d-717d-45cf-bbd9-48c4ba78de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:04.648883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685ee4583a0b45449a23c2769780c012'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.650087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685ee4583a0b45449a23c2769780c012'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.652690 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685ee4583a0b45449a23c2769780c012'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:04.685978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 831/1661] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:04 2022] DELETE /identity/v3/users/685ee4583a0b45449a23c2769780c012 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:04.724552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 831/1662] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:04.725799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:04.785661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:04.882929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da75af-8aae-4a10-b2d9-14f21734c71a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:04.895781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c00d6-6b70-4e88-99c2-5d3cb9df2903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:04.897343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c00d6-6b70-4e88-99c2-5d3cb9df2903 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:04.897343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c00d6-6b70-4e88-99c2-5d3cb9df2903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:04.897343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c00d6-6b70-4e88-99c2-5d3cb9df2903 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:04.897646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c00d6-6b70-4e88-99c2-5d3cb9df2903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:04.898013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c00d6-6b70-4e88-99c2-5d3cb9df2903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:04.903008 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c00d6-6b70-4e88-99c2-5d3cb9df2903 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:04.934406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 832/1663] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:04.938793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93352d-717d-45cf-bbd9-48c4ba78de66 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:04.958675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93352d-717d-45cf-bbd9-48c4ba78de66 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:04.976237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9794a8-ede3-439f-8a6b-32706720bf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:04.976963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9794a8-ede3-439f-8a6b-32706720bf6d admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:04.977323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9794a8-ede3-439f-8a6b-32706720bf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:04.977667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9794a8-ede3-439f-8a6b-32706720bf6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:04.978377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9794a8-ede3-439f-8a6b-32706720bf6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:04.980399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9794a8-ede3-439f-8a6b-32706720bf6d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:05.049632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 832/1664] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:04 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:05.053950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c00d6-6b70-4e88-99c2-5d3cb9df2903 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:05.068195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c00d6-6b70-4e88-99c2-5d3cb9df2903 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:05.204093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79047c-8cb5-4ea0-a848-3a511f0e44ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:05.205685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79047c-8cb5-4ea0-a848-3a511f0e44ae admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:05.206180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79047c-8cb5-4ea0-a848-3a511f0e44ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:05.206696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79047c-8cb5-4ea0-a848-3a511f0e44ae admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:05.207631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79047c-8cb5-4ea0-a848-3a511f0e44ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:05.211413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79047c-8cb5-4ea0-a848-3a511f0e44ae admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:05.288797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 833/1665] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:05 2022] POST /identity/v3/users => generated 285 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:05.289817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9794a8-ede3-439f-8a6b-32706720bf6d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:05.305354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9794a8-ede3-439f-8a6b-32706720bf6d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:05.319750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73454744-4a3f-4eed-a711-5925540e7b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:05.321046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73454744-4a3f-4eed-a711-5925540e7b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:05.321482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73454744-4a3f-4eed-a711-5925540e7b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:05.321909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73454744-4a3f-4eed-a711-5925540e7b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:05.322717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73454744-4a3f-4eed-a711-5925540e7b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:05.330087 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73454744-4a3f-4eed-a711-5925540e7b96 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:05.348790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:05.375872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79047c-8cb5-4ea0-a848-3a511f0e44ae admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:05.388918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79047c-8cb5-4ea0-a848-3a511f0e44ae admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:05.393257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 833/1666] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:05.397868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b30c3-6a12-4ac2-ba18-cb5c47201191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:05.399462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b30c3-6a12-4ac2-ba18-cb5c47201191 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:05.399917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b30c3-6a12-4ac2-ba18-cb5c47201191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:05.400162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b30c3-6a12-4ac2-ba18-cb5c47201191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:05.400959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b30c3-6a12-4ac2-ba18-cb5c47201191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:05.403947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b30c3-6a12-4ac2-ba18-cb5c47201191 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:05.432422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 834/1667] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:05.433378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73454744-4a3f-4eed-a711-5925540e7b96 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:05.446448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73454744-4a3f-4eed-a711-5925540e7b96 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:05.459250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11625876-00c0-4d0f-bd0e-3d1b030fb961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:05.459937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11625876-00c0-4d0f-bd0e-3d1b030fb961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:05.460233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11625876-00c0-4d0f-bd0e-3d1b030fb961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:05.460489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11625876-00c0-4d0f-bd0e-3d1b030fb961 admin admin] PATH_INFO: `/v3/projects/f7e4eb3676b64434894e2238e9635b66/users/a8c837faa7a8414aa5068f1218f3949c/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:05.497635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11625876-00c0-4d0f-bd0e-3d1b030fb961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4eb3676b64434894e2238e9635b66, user_id=a8c837faa7a8414aa5068f1218f3949c, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:05.502433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11625876-00c0-4d0f-bd0e-3d1b030fb961 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:05.544981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1af328-7e01-42bd-bf42-9ffdaf03f2ce None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:05.545628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1af328-7e01-42bd-bf42-9ffdaf03f2ce None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:05.546195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1af328-7e01-42bd-bf42-9ffdaf03f2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:05.571360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 834/1668] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:05 2022] PUT /identity/v3/projects/f7e4eb3676b64434894e2238e9635b66/users/a8c837faa7a8414aa5068f1218f3949c/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:05.616004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bf1af328-7e01-42bd-bf42-9ffdaf03f2ce None None] MFA Rules not processed for user `a8c837faa7a8414aa5068f1218f3949c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:05.617714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bf1af328-7e01-42bd-bf42-9ffdaf03f2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:05.764120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11625876-00c0-4d0f-bd0e-3d1b030fb961 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:05.769244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 835/1669] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:05.927753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11625876-00c0-4d0f-bd0e-3d1b030fb961 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:05.934598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2826dd4-f6f5-42df-b972-b616a63fd9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:05.936001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2826dd4-f6f5-42df-b972-b616a63fd9ba admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:05.936435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2826dd4-f6f5-42df-b972-b616a63fd9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:05.936845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2826dd4-f6f5-42df-b972-b616a63fd9ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:05.937656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2826dd4-f6f5-42df-b972-b616a63fd9ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:05.940732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2826dd4-f6f5-42df-b972-b616a63fd9ba admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:06.005098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 835/1670] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:05 2022] POST /identity/v3/projects => generated 367 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:06.011446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1af328-7e01-42bd-bf42-9ffdaf03f2ce None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:06.050023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1af328-7e01-42bd-bf42-9ffdaf03f2ce None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:06.122599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:06.212912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe5b68-ba0d-49cb-94a8-ae4416972c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:06.213691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe5b68-ba0d-49cb-94a8-ae4416972c63 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:06.214072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe5b68-ba0d-49cb-94a8-ae4416972c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:06.214447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe5b68-ba0d-49cb-94a8-ae4416972c63 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:06.215237 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe5b68-ba0d-49cb-94a8-ae4416972c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:06.217332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe5b68-ba0d-49cb-94a8-ae4416972c63 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:06.328905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 836/1671] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:06 2022] POST /identity/v3/users => generated 290 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:06.330425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2826dd4-f6f5-42df-b972-b616a63fd9ba admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:06.351929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2826dd4-f6f5-42df-b972-b616a63fd9ba admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:06.370826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b50501-be34-4d0f-b74a-32a31f06abb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:06.371487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b50501-be34-4d0f-b74a-32a31f06abb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:06.372102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b50501-be34-4d0f-b74a-32a31f06abb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:06.375636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b50501-be34-4d0f-b74a-32a31f06abb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:06.376342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b50501-be34-4d0f-b74a-32a31f06abb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:06.380045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b50501-be34-4d0f-b74a-32a31f06abb2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:06.429507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 836/1672] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:06.430813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe5b68-ba0d-49cb-94a8-ae4416972c63 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:06.453259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe5b68-ba0d-49cb-94a8-ae4416972c63 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:06.466649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af39cbc-3b37-45d8-a46c-491f24d8c937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:06.467902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af39cbc-3b37-45d8-a46c-491f24d8c937 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:06.468179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af39cbc-3b37-45d8-a46c-491f24d8c937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:06.468450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af39cbc-3b37-45d8-a46c-491f24d8c937 admin admin] PATH_INFO: `/v3/projects/ea21ae02cdc749b8aac85786da26260f/users/2fa32f52833d4f5498238a56a5c4a8c6/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:06.534832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af39cbc-3b37-45d8-a46c-491f24d8c937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea21ae02cdc749b8aac85786da26260f, user_id=2fa32f52833d4f5498238a56a5c4a8c6, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:06.537578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af39cbc-3b37-45d8-a46c-491f24d8c937 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:06.582693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 837/1673] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:06 2022] PUT /identity/v3/projects/ea21ae02cdc749b8aac85786da26260f/users/2fa32f52833d4f5498238a56a5c4a8c6/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:06.586702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b50501-be34-4d0f-b74a-32a31f06abb2 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:06.598094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b50501-be34-4d0f-b74a-32a31f06abb2 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:06.715808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af39cbc-3b37-45d8-a46c-491f24d8c937 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:06.718199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2af39cbc-3b37-45d8-a46c-491f24d8c937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:06.876559 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5331295-8a84-48f3-a3e1-4e96148ae576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:06.877384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5331295-8a84-48f3-a3e1-4e96148ae576 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:06.878357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5331295-8a84-48f3-a3e1-4e96148ae576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:06.880702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5331295-8a84-48f3-a3e1-4e96148ae576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:06.883233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5331295-8a84-48f3-a3e1-4e96148ae576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:06.889765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5331295-8a84-48f3-a3e1-4e96148ae576 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:06.923939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af39cbc-3b37-45d8-a46c-491f24d8c937 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:06.932810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 837/1674] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:06.934481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5331295-8a84-48f3-a3e1-4e96148ae576 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:06.934614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:06.935907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:06.936236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:06.936509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] PATH_INFO: `/v3/projects/dd3a2092704a491cbfa3ba054f1f8f49` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:06.939251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd3a2092704a491cbfa3ba054f1f8f49)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:06.941187 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:06.959494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5331295-8a84-48f3-a3e1-4e96148ae576 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:06.973375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacd49e-c19f-405d-b4de-2585eb985ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:06.976407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacd49e-c19f-405d-b4de-2585eb985ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:06.976958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacd49e-c19f-405d-b4de-2585eb985ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:06.977439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacd49e-c19f-405d-b4de-2585eb985ea9 admin admin] PATH_INFO: `/v3/projects/ea21ae02cdc749b8aac85786da26260f/users/2fa32f52833d4f5498238a56a5c4a8c6/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:06.980251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacd49e-c19f-405d-b4de-2585eb985ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea21ae02cdc749b8aac85786da26260f, user_id=2fa32f52833d4f5498238a56a5c4a8c6, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:06.985612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacd49e-c19f-405d-b4de-2585eb985ea9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:07.011979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 838/1675] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:06 2022] PUT /identity/v3/projects/ea21ae02cdc749b8aac85786da26260f/users/2fa32f52833d4f5498238a56a5c4a8c6/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:07.013789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea694b-74e2-4996-a93c-38c7640f5a36 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:07.014423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea694b-74e2-4996-a93c-38c7640f5a36 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:07.014805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea694b-74e2-4996-a93c-38c7640f5a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:07.071749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-adea694b-74e2-4996-a93c-38c7640f5a36 None None] MFA Rules not processed for user `2fa32f52833d4f5498238a56a5c4a8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:07.075418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-adea694b-74e2-4996-a93c-38c7640f5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:07.100059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:07.111722 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd3a2092704a491cbfa3ba054f1f8f49', 'request_id': 'req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:07.122618 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd3a2092704a491cbfa3ba054f1f8f49', 'request_id': 'req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:07.140048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd3a2092704a491cbfa3ba054f1f8f49', 'request_id': 'req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:07.177563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] The token cache is being invalidate because project dd3a2092704a491cbfa3ba054f1f8f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:07.178845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:07.180298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-baf9a000-7ccf-48aa-a8b8-857ade0d2ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:07.184306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 838/1676] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:06 2022] DELETE /identity/v3/projects/dd3a2092704a491cbfa3ba054f1f8f49 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:07.199125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b91de-7657-42ec-a892-bd96ff09d1f6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:07.199766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b91de-7657-42ec-a892-bd96ff09d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:07.200447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b91de-7657-42ec-a892-bd96ff09d1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:07.230503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 839/1677] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:07.246803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea694b-74e2-4996-a93c-38c7640f5a36 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:07.248693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-adea694b-74e2-4996-a93c-38c7640f5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:07.266098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1e7b91de-7657-42ec-a892-bd96ff09d1f6 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:07.286941 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1e7b91de-7657-42ec-a892-bd96ff09d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:07.340496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:07.452657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea694b-74e2-4996-a93c-38c7640f5a36 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:07.463774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22ea2-889c-4df2-a482-a0f5d342b11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:07.472622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22ea2-889c-4df2-a482-a0f5d342b11f service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:07.473561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22ea2-889c-4df2-a482-a0f5d342b11f service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:07.475518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22ea2-889c-4df2-a482-a0f5d342b11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:07.508898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d22ea2-889c-4df2-a482-a0f5d342b11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:07.515220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d22ea2-889c-4df2-a482-a0f5d342b11f service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:07.527105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 839/1678] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:07.532248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71614103-3702-46f9-a775-9d93ed80735b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:07.534410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71614103-3702-46f9-a775-9d93ed80735b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:07.535414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71614103-3702-46f9-a775-9d93ed80735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:07.595414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 840/1679] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:07.631211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-71614103-3702-46f9-a775-9d93ed80735b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:07.633139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-71614103-3702-46f9-a775-9d93ed80735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:07.688422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 840/1680] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:07.697168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22ea2-889c-4df2-a482-a0f5d342b11f service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:07.712225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71614103-3702-46f9-a775-9d93ed80735b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:07.719156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22ea2-889c-4df2-a482-a0f5d342b11f service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:07.731511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915b57c-1d20-4065-ae74-60818f1745a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:07.735047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915b57c-1d20-4065-ae74-60818f1745a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:07.735047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915b57c-1d20-4065-ae74-60818f1745a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:07.735047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915b57c-1d20-4065-ae74-60818f1745a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:07.748053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71614103-3702-46f9-a775-9d93ed80735b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:07.752695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915b57c-1d20-4065-ae74-60818f1745a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:07.753020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915b57c-1d20-4065-ae74-60818f1745a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:07.755598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915b57c-1d20-4065-ae74-60818f1745a4 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:07.766093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca6070-bbfc-4ef2-8e03-fc06702d320a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:07.766721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca6070-bbfc-4ef2-8e03-fc06702d320a service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:07.767016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca6070-bbfc-4ef2-8e03-fc06702d320a service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:07.767286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca6070-bbfc-4ef2-8e03-fc06702d320a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:07.777308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-17ca6070-bbfc-4ef2-8e03-fc06702d320a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:07.804988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 841/1681] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:07.810212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915b57c-1d20-4065-ae74-60818f1745a4 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:07.851702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915b57c-1d20-4065-ae74-60818f1745a4 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:07.865635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726ae7d-5360-4d33-83ed-ee1c8d48f7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:07.866422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726ae7d-5360-4d33-83ed-ee1c8d48f7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:07.872171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726ae7d-5360-4d33-83ed-ee1c8d48f7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:07.872547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726ae7d-5360-4d33-83ed-ee1c8d48f7a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:07.875021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726ae7d-5360-4d33-83ed-ee1c8d48f7a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:07.884315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726ae7d-5360-4d33-83ed-ee1c8d48f7a1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:07.980556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca6070-bbfc-4ef2-8e03-fc06702d320a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:07.983191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 842/1682] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:07 2022] POST /identity/v3/projects => generated 389 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:07.986641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726ae7d-5360-4d33-83ed-ee1c8d48f7a1 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:07.989165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca6070-bbfc-4ef2-8e03-fc06702d320a service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:08.020248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 841/1683] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:08.139499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726ae7d-5360-4d33-83ed-ee1c8d48f7a1 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:08.145320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9060361-9008-49b6-bf16-8bf97cf01fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:08.146061 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060361-9008-49b6-bf16-8bf97cf01fae admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:08.148405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060361-9008-49b6-bf16-8bf97cf01fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:08.148929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060361-9008-49b6-bf16-8bf97cf01fae admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:08.149602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9060361-9008-49b6-bf16-8bf97cf01fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:08.151943 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9060361-9008-49b6-bf16-8bf97cf01fae admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:08.224738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 843/1684] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:07 2022] POST /identity/v3/users => generated 295 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:08.229637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca6070-bbfc-4ef2-8e03-fc06702d320a service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:08.256763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca6070-bbfc-4ef2-8e03-fc06702d320a service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:08.281201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae36c-c0cf-418e-ba12-415cbc33c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:08.282222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae36c-c0cf-418e-ba12-415cbc33c5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:08.282688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae36c-c0cf-418e-ba12-415cbc33c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:08.283245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae36c-c0cf-418e-ba12-415cbc33c5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:08.284120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae36c-c0cf-418e-ba12-415cbc33c5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:08.292951 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae36c-c0cf-418e-ba12-415cbc33c5cf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:08.338723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 842/1685] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:08.344213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9060361-9008-49b6-bf16-8bf97cf01fae admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:08.361267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9060361-9008-49b6-bf16-8bf97cf01fae admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:08.375923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64b4f7-60cd-49d6-bb69-a2da91a255dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:08.376551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64b4f7-60cd-49d6-bb69-a2da91a255dc admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:08.376828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64b4f7-60cd-49d6-bb69-a2da91a255dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:08.377256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64b4f7-60cd-49d6-bb69-a2da91a255dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:08.377921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64b4f7-60cd-49d6-bb69-a2da91a255dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:08.379567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64b4f7-60cd-49d6-bb69-a2da91a255dc admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:08.413968 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 844/1686] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:08.418740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae36c-c0cf-418e-ba12-415cbc33c5cf admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:08.436977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae36c-c0cf-418e-ba12-415cbc33c5cf admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:08.468293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc856a-cd59-41f9-accf-19eb8dbcac25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:08.468975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc856a-cd59-41f9-accf-19eb8dbcac25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:08.469261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc856a-cd59-41f9-accf-19eb8dbcac25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:08.469495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc856a-cd59-41f9-accf-19eb8dbcac25 admin admin] PATH_INFO: `/v3/projects/2e404e3752184b5c8bb8449f88db7573/users/2db3ed89bc2a40c6958fbc782bd68138/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:08.526869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc856a-cd59-41f9-accf-19eb8dbcac25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e404e3752184b5c8bb8449f88db7573, user_id=2db3ed89bc2a40c6958fbc782bd68138, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:08.530267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc856a-cd59-41f9-accf-19eb8dbcac25 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:08.577844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 843/1687] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:08 2022] PUT /identity/v3/projects/2e404e3752184b5c8bb8449f88db7573/users/2db3ed89bc2a40c6958fbc782bd68138/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:08.591248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede3885-de82-4615-af68-348380aa09e6 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:08.591755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede3885-de82-4615-af68-348380aa09e6 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:08.592758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede3885-de82-4615-af68-348380aa09e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:08.605309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:08.696772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-dede3885-de82-4615-af68-348380aa09e6 None None] MFA Rules not processed for user `2db3ed89bc2a40c6958fbc782bd68138`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:08.702458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-dede3885-de82-4615-af68-348380aa09e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:08.757555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0715e-7e67-46e2-b32e-434c42dfd1cb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:08.757930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0715e-7e67-46e2-b32e-434c42dfd1cb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:08.758110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0715e-7e67-46e2-b32e-434c42dfd1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:08.836516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-acc0715e-7e67-46e2-b32e-434c42dfd1cb None None] MFA Rules not processed for user `0d53857c736c4a75abe0d4979517bc58`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:08.837979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-acc0715e-7e67-46e2-b32e-434c42dfd1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:08.899967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 845/1688] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:08.902330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede3885-de82-4615-af68-348380aa09e6 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:08.960362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 844/1689] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:08.964518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b3517-0d6a-4dea-a8c5-76eda50cef1e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:08.965470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b3517-0d6a-4dea-a8c5-76eda50cef1e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:08.966020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b3517-0d6a-4dea-a8c5-76eda50cef1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:09.017865 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:09.039848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3d3b3517-0d6a-4dea-a8c5-76eda50cef1e None None] MFA Rules not processed for user `49e4e90c2c944fabb77bfe1e90d4f4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:09.048086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3d3b3517-0d6a-4dea-a8c5-76eda50cef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:09.076215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede3885-de82-4615-af68-348380aa09e6 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:09.090553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407e4a3-885b-4a9a-bf59-e878ce5af31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:09.091275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407e4a3-885b-4a9a-bf59-e878ce5af31a admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:09.091559 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407e4a3-885b-4a9a-bf59-e878ce5af31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:09.091837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407e4a3-885b-4a9a-bf59-e878ce5af31a admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:09.093292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407e4a3-885b-4a9a-bf59-e878ce5af31a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:09.096024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407e4a3-885b-4a9a-bf59-e878ce5af31a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:09.144189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 846/1690] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:08 2022] POST /identity/v3/projects => generated 387 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:09.145847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222cfd8-3a79-4bb1-880f-137888a01f9a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:09.146152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222cfd8-3a79-4bb1-880f-137888a01f9a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:09.147401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222cfd8-3a79-4bb1-880f-137888a01f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:09.185279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0222cfd8-3a79-4bb1-880f-137888a01f9a None None] MFA Rules not processed for user `0d53857c736c4a75abe0d4979517bc58`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:09.188764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0222cfd8-3a79-4bb1-880f-137888a01f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:09.248829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 845/1691] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:08 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:09.250537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b3517-0d6a-4dea-a8c5-76eda50cef1e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:09.293265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b3517-0d6a-4dea-a8c5-76eda50cef1e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:09.359649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 847/1692] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:09.372734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222cfd8-3a79-4bb1-880f-137888a01f9a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:09.376271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0222cfd8-3a79-4bb1-880f-137888a01f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:09.421688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:09.506726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47c963-33a7-448f-9d82-59d64e275597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:09.509504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47c963-33a7-448f-9d82-59d64e275597 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:09.509802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47c963-33a7-448f-9d82-59d64e275597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:09.510180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47c963-33a7-448f-9d82-59d64e275597 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:09.512423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47c963-33a7-448f-9d82-59d64e275597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:09.520671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47c963-33a7-448f-9d82-59d64e275597 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:09.537721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222cfd8-3a79-4bb1-880f-137888a01f9a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:09.551105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e030178-e31f-45ec-9e7f-03fb318ba743 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:09.551937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030178-e31f-45ec-9e7f-03fb318ba743 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:09.552214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030178-e31f-45ec-9e7f-03fb318ba743 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:09.552487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030178-e31f-45ec-9e7f-03fb318ba743 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:09.569457 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e030178-e31f-45ec-9e7f-03fb318ba743 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:09.571932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e030178-e31f-45ec-9e7f-03fb318ba743 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:09.590192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 848/1693] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:09.593539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 846/1694] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:09 2022] POST /identity/v3/users => generated 300 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:09.597887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e030178-e31f-45ec-9e7f-03fb318ba743 service nova] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:09.617997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e030178-e31f-45ec-9e7f-03fb318ba743 service nova] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:09.625875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b97351-9bf2-422b-9a66-4b75307a92ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:09.626506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b97351-9bf2-422b-9a66-4b75307a92ca admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:09.626785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b97351-9bf2-422b-9a66-4b75307a92ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:09.627116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b97351-9bf2-422b-9a66-4b75307a92ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:09.627921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b97351-9bf2-422b-9a66-4b75307a92ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:09.631754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b97351-9bf2-422b-9a66-4b75307a92ca admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:09.647277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 849/1695] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:09.647692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47c963-33a7-448f-9d82-59d64e275597 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:09.660730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47c963-33a7-448f-9d82-59d64e275597 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:09.672427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f87e14-c998-4392-9e1a-c829e1ee1d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:09.673128 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f87e14-c998-4392-9e1a-c829e1ee1d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:09.673396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f87e14-c998-4392-9e1a-c829e1ee1d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:09.673672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f87e14-c998-4392-9e1a-c829e1ee1d8d admin admin] PATH_INFO: `/v3/projects/b10437222ace4a8091f5cc7c49898339/users/8fbd19314824451bac1dce24dc31d38f/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:09.710099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f87e14-c998-4392-9e1a-c829e1ee1d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10437222ace4a8091f5cc7c49898339, user_id=8fbd19314824451bac1dce24dc31d38f, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:09.715460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f87e14-c998-4392-9e1a-c829e1ee1d8d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:09.764320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 847/1696] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:09 2022] PUT /identity/v3/projects/b10437222ace4a8091f5cc7c49898339/users/8fbd19314824451bac1dce24dc31d38f/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:09.769765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b97351-9bf2-422b-9a66-4b75307a92ca admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:09.813910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b97351-9bf2-422b-9a66-4b75307a92ca admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:09.849772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a90cbe-4dff-4891-a8da-c3113e6bbe14 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:09.850099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a90cbe-4dff-4891-a8da-c3113e6bbe14 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:09.850349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a90cbe-4dff-4891-a8da-c3113e6bbe14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:09.952652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-01a90cbe-4dff-4891-a8da-c3113e6bbe14 None None] MFA Rules not processed for user `2dd3653b931643318f1a1b0820620277`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:09.953911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-01a90cbe-4dff-4891-a8da-c3113e6bbe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:10.030531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15dd4d-dd5c-429f-b06c-c472017d63e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:10.033778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15dd4d-dd5c-429f-b06c-c472017d63e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:10.033778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15dd4d-dd5c-429f-b06c-c472017d63e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:10.033997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15dd4d-dd5c-429f-b06c-c472017d63e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:10.034648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15dd4d-dd5c-429f-b06c-c472017d63e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:10.041260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15dd4d-dd5c-429f-b06c-c472017d63e7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:10.088670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 850/1697] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:10.091136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15dd4d-dd5c-429f-b06c-c472017d63e7 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:10.093038 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:10.154136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 848/1698] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:10.155225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a90cbe-4dff-4891-a8da-c3113e6bbe14 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:10.157415 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-01a90cbe-4dff-4891-a8da-c3113e6bbe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:10.159426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15dd4d-dd5c-429f-b06c-c472017d63e7 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:10.171480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e58fc-70fb-4cd7-9509-0752feafebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:10.172157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e58fc-70fb-4cd7-9509-0752feafebd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:10.172453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e58fc-70fb-4cd7-9509-0752feafebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:10.172719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e58fc-70fb-4cd7-9509-0752feafebd2 admin admin] PATH_INFO: `/v3/projects/b10437222ace4a8091f5cc7c49898339/users/8fbd19314824451bac1dce24dc31d38f/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:10.187401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e58fc-70fb-4cd7-9509-0752feafebd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10437222ace4a8091f5cc7c49898339, user_id=8fbd19314824451bac1dce24dc31d38f, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:10.187401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e58fc-70fb-4cd7-9509-0752feafebd2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:10.239029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 851/1699] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:10 2022] PUT /identity/v3/projects/b10437222ace4a8091f5cc7c49898339/users/8fbd19314824451bac1dce24dc31d38f/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:10.251310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e66c82-36ea-45da-8190-93bc335b0d6f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:10.253603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e66c82-36ea-45da-8190-93bc335b0d6f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:10.254081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e66c82-36ea-45da-8190-93bc335b0d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:10.340636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c6e66c82-36ea-45da-8190-93bc335b0d6f None None] MFA Rules not processed for user `8fbd19314824451bac1dce24dc31d38f`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:10.341884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c6e66c82-36ea-45da-8190-93bc335b0d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:10.432212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a90cbe-4dff-4891-a8da-c3113e6bbe14 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:10.532807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 852/1700] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:10.612645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6bd07-4d6f-46ab-ad6e-7d3bbaf199af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:10.613443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6bd07-4d6f-46ab-ad6e-7d3bbaf199af service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:10.613926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6bd07-4d6f-46ab-ad6e-7d3bbaf199af service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:10.614212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6bd07-4d6f-46ab-ad6e-7d3bbaf199af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:10.713707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e66c82-36ea-45da-8190-93bc335b0d6f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:10.730397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:10.744970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6bd07-4d6f-46ab-ad6e-7d3bbaf199af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:10.750412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6bd07-4d6f-46ab-ad6e-7d3bbaf199af service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:10.804381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 849/1701] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:10.806090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61364356-493d-4ae7-a76d-f37117a0f9b9 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:10.807712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61364356-493d-4ae7-a76d-f37117a0f9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:10.811623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61364356-493d-4ae7-a76d-f37117a0f9b9 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:10.823279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 850/1702] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 10 10:03:10 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:03:10.831540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b5cc0-b7ea-402b-8864-731d38370dab None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:10.832290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b5cc0-b7ea-402b-8864-731d38370dab None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:10.833391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b5cc0-b7ea-402b-8864-731d38370dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:10.904918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e66c82-36ea-45da-8190-93bc335b0d6f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:10.911084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309a81e-012b-43db-8024-e8910ba2311c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:10.911887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309a81e-012b-43db-8024-e8910ba2311c service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:10.912947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309a81e-012b-43db-8024-e8910ba2311c service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:10.913161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309a81e-012b-43db-8024-e8910ba2311c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:10.926193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309a81e-012b-43db-8024-e8910ba2311c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:10.927989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309a81e-012b-43db-8024-e8910ba2311c service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:10.989951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 853/1703] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:11.058397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-be3b5cc0-b7ea-402b-8864-731d38370dab None None] MFA Rules not processed for user `ab9b7576a3834862af6abb7e0416f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:11.063864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-be3b5cc0-b7ea-402b-8864-731d38370dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:11.089220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:11.092027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309a81e-012b-43db-8024-e8910ba2311c service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:11.093729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e309a81e-012b-43db-8024-e8910ba2311c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:11.220548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 851/1704] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:03:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:11.337227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b5cc0-b7ea-402b-8864-731d38370dab None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:11.350396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-be3b5cc0-b7ea-402b-8864-731d38370dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:11.368982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b5cc0-b7ea-402b-8864-731d38370dab None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:11.374308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcda73f-f303-4210-b6d8-0bd46d1e0402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:11.375569 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcda73f-f303-4210-b6d8-0bd46d1e0402 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:11.375990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcda73f-f303-4210-b6d8-0bd46d1e0402 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:11.376395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcda73f-f303-4210-b6d8-0bd46d1e0402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:11.386040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcda73f-f303-4210-b6d8-0bd46d1e0402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:11.387678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcda73f-f303-4210-b6d8-0bd46d1e0402 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:11.414543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309a81e-012b-43db-8024-e8910ba2311c service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:11.416668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 852/1705] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:11.439996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:11.443625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:11.445920 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:11.446210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] PATH_INFO: `/v3/users/ae0020c2bbf54f969b814a4c6fc77bf3` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:11.451937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0020c2bbf54f969b814a4c6fc77bf3)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:11.457441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:11.476111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ed11d-f779-47d9-89e0-343e76604431 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:11.478987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ed11d-f779-47d9-89e0-343e76604431 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:11.479518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ed11d-f779-47d9-89e0-343e76604431 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:11.482035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 853/1706] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 10:03:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:03:11.539830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d47be9-f55d-4001-8af3-60cd0aa340e8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:11.540664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d47be9-f55d-4001-8af3-60cd0aa340e8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:11.541300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d47be9-f55d-4001-8af3-60cd0aa340e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:11.557768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0020c2bbf54f969b814a4c6fc77bf3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.568308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0020c2bbf54f969b814a4c6fc77bf3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.579176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0020c2bbf54f969b814a4c6fc77bf3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.594205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d8d47be9-f55d-4001-8af3-60cd0aa340e8 None None] MFA Rules not processed for user `d3d7b3cb498a4da18e203456e09b59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:11.594979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0020c2bbf54f969b814a4c6fc77bf3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.595772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0020c2bbf54f969b814a4c6fc77bf3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.600661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d8d47be9-f55d-4001-8af3-60cd0aa340e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:11.606290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 854/1707] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:11 2022] DELETE /identity/v3/users/ae0020c2bbf54f969b814a4c6fc77bf3 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:11.608620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:11.612169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:11.651414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:11.708775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754afe8d-ea45-4d2d-9ecb-6417f2622b3c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:11.716730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:11.718003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:11.718288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:11.718713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] PATH_INFO: `/v3/users/2db3ed89bc2a40c6958fbc782bd68138` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:11.720111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db3ed89bc2a40c6958fbc782bd68138)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:11.722406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:11.773947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 854/1708] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 10:03:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:11.775533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52c779-a848-4f92-84bc-090fbdd1c5c0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:11.775914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52c779-a848-4f92-84bc-090fbdd1c5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:11.776563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52c779-a848-4f92-84bc-090fbdd1c5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:11.815242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db3ed89bc2a40c6958fbc782bd68138'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.828967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db3ed89bc2a40c6958fbc782bd68138'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.829139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2c52c779-a848-4f92-84bc-090fbdd1c5c0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:11.829981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2c52c779-a848-4f92-84bc-090fbdd1c5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:11.844925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db3ed89bc2a40c6958fbc782bd68138'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.855118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 855/1709] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:11.856544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d0a63-9949-425f-ad4e-94f6334ab634 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:11.857071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d0a63-9949-425f-ad4e-94f6334ab634 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:11.857290 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d0a63-9949-425f-ad4e-94f6334ab634 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:11.859579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 856/1710] 10.222.0.20 () {54 vars in 961 bytes} [Wed Aug 10 10:03:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:03:11.867467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9dd3-9bd4-46bb-9472-8771a1338d84 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:11.867974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9dd3-9bd4-46bb-9472-8771a1338d84 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:11.868349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9dd3-9bd4-46bb-9472-8771a1338d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:11.882582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db3ed89bc2a40c6958fbc782bd68138'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.888182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db3ed89bc2a40c6958fbc782bd68138'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:11.908245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 855/1711] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:11 2022] DELETE /identity/v3/users/2db3ed89bc2a40c6958fbc782bd68138 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:11.909222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:11.910647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:11.916999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-13ca9dd3-9bd4-46bb-9472-8771a1338d84 None None] MFA Rules not processed for user `d3d7b3cb498a4da18e203456e09b59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:11.921061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-13ca9dd3-9bd4-46bb-9472-8771a1338d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:12.074106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a209e38-90d4-44dd-bedd-1f736ffa5f14 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:12.083844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 857/1712] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Aug 10 10:03:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:12.087752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36202c-709a-4252-800d-80fc51fa51ee None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:12.088149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36202c-709a-4252-800d-80fc51fa51ee None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:12.089146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805e7ee-62d9-4589-a627-03fb9f9cc745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:12.089594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36202c-709a-4252-800d-80fc51fa51ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:12.090698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805e7ee-62d9-4589-a627-03fb9f9cc745 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:12.092725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805e7ee-62d9-4589-a627-03fb9f9cc745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:12.093741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805e7ee-62d9-4589-a627-03fb9f9cc745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:12.109793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6805e7ee-62d9-4589-a627-03fb9f9cc745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:12.152823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5a36202c-709a-4252-800d-80fc51fa51ee None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:12.157387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5a36202c-709a-4252-800d-80fc51fa51ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:12.246660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:12.335656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 858/1713] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:12.338562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36202c-709a-4252-800d-80fc51fa51ee None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:12.359307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:12.364795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805e7ee-62d9-4589-a627-03fb9f9cc745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:12.373431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805e7ee-62d9-4589-a627-03fb9f9cc745 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:12.384840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36202c-709a-4252-800d-80fc51fa51ee None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:12.393122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 856/1714] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:12.428227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:12.428522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:12.428812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:12.429091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:12.433290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805e7ee-62d9-4589-a627-03fb9f9cc745 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:12.435434 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:12.443651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805e7ee-62d9-4589-a627-03fb9f9cc745 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:12.457282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876cfae-0d1a-4949-be0c-b7cc039865ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:12.457868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876cfae-0d1a-4949-be0c-b7cc039865ae service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:12.458231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876cfae-0d1a-4949-be0c-b7cc039865ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:12.458544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876cfae-0d1a-4949-be0c-b7cc039865ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:12.473066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:12.474742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:12.496839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 859/1715] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:12.511765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876cfae-0d1a-4949-be0c-b7cc039865ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:12.515030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876cfae-0d1a-4949-be0c-b7cc039865ae service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:12.537671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:12.539192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:12.547946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 857/1716] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:12.574921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d5891-1127-489f-a31f-a29101b2548a service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:12.605803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:12.606442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:12.606725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:12.607021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:12.633520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:12.672141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:12.674318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:12.701162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 860/1717] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:13.595417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623ae45-f4a4-4268-88fb-a75c976a2ad1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:13.595925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623ae45-f4a4-4268-88fb-a75c976a2ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:13.596165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623ae45-f4a4-4268-88fb-a75c976a2ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:13.662842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:13.664681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:13.666322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-4623ae45-f4a4-4268-88fb-a75c976a2ad1 None None] MFA Rules not processed for user `befbca3738f0481296b63c70d6d2a797`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:13.672976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4623ae45-f4a4-4268-88fb-a75c976a2ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:13.693208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9315711-5499-4c3d-a7db-b6fabe22638e service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:13.705624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:13.706418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:13.706608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:13.706888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] PATH_INFO: `/v3/users/32fdacb7de1b46aea9a9b515b0a51fbe` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:13.708219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fdacb7de1b46aea9a9b515b0a51fbe)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:13.710119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:13.816413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fdacb7de1b46aea9a9b515b0a51fbe'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:13.845496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fdacb7de1b46aea9a9b515b0a51fbe'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:13.897695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fdacb7de1b46aea9a9b515b0a51fbe'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:13.900851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fdacb7de1b46aea9a9b515b0a51fbe'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:13.905373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d01d6fd8-ed49-42ed-b5d2-a171c571fb92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fdacb7de1b46aea9a9b515b0a51fbe'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:13.912260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 858/1718] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:13.913252 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623ae45-f4a4-4268-88fb-a75c976a2ad1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:13.923690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4623ae45-f4a4-4268-88fb-a75c976a2ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:13.951056 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 861/1719] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:13 2022] DELETE /identity/v3/users/32fdacb7de1b46aea9a9b515b0a51fbe => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:13.956126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2c149-1ffc-40d8-9b28-5f3d13cc92d7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:13.956417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2c149-1ffc-40d8-9b28-5f3d13cc92d7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:13.958429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2c149-1ffc-40d8-9b28-5f3d13cc92d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:13.971517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623ae45-f4a4-4268-88fb-a75c976a2ad1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:14.047017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5ea2c149-1ffc-40d8-9b28-5f3d13cc92d7 None None] MFA Rules not processed for user `befbca3738f0481296b63c70d6d2a797`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:14.048469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5ea2c149-1ffc-40d8-9b28-5f3d13cc92d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:14.217828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:14.218908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:14.219265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:14.219545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] PATH_INFO: `/v3/users/a8c837faa7a8414aa5068f1218f3949c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:14.220943 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c837faa7a8414aa5068f1218f3949c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:14.223892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 862/1720] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:14.224404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:14.230564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2c149-1ffc-40d8-9b28-5f3d13cc92d7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:14.232949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5ea2c149-1ffc-40d8-9b28-5f3d13cc92d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:14.336019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:14.410513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c837faa7a8414aa5068f1218f3949c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:14.434988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c837faa7a8414aa5068f1218f3949c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:14.466676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c837faa7a8414aa5068f1218f3949c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:14.477423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2c149-1ffc-40d8-9b28-5f3d13cc92d7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:14.495640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c47c8-2be2-4764-806d-c58cb7145c1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:14.496278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c47c8-2be2-4764-806d-c58cb7145c1a service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:14.496557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c47c8-2be2-4764-806d-c58cb7145c1a service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:14.496831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c47c8-2be2-4764-806d-c58cb7145c1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:14.504520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9e2c47c8-2be2-4764-806d-c58cb7145c1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:14.508727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:14.518938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c837faa7a8414aa5068f1218f3949c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:14.521949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c837faa7a8414aa5068f1218f3949c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:14.531901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c47c8-2be2-4764-806d-c58cb7145c1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:14.535855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c47c8-2be2-4764-806d-c58cb7145c1a service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:14.540252 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 859/1721] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:13 2022] DELETE /identity/v3/users/a8c837faa7a8414aa5068f1218f3949c => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:14.542214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:14.547275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:14.580437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 863/1722] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:14.583836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4e837-ba31-40b8-9814-20effe6a6b84 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:14.584126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4e837-ba31-40b8-9814-20effe6a6b84 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:14.587950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4e837-ba31-40b8-9814-20effe6a6b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:14.658155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-61d4e837-ba31-40b8-9814-20effe6a6b84 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:14.660045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-61d4e837-ba31-40b8-9814-20effe6a6b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:14.813624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 864/1723] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:14.814272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054339-4792-45ee-b8b2-068bc7301f6c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:14.815960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4e837-ba31-40b8-9814-20effe6a6b84 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:14.827064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-61d4e837-ba31-40b8-9814-20effe6a6b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:14.834519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:14.835243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:14.835521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:14.835795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] PATH_INFO: `/v3/projects/b6f12c730fb242e9868838c412572e0e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:14.837207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f12c730fb242e9868838c412572e0e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:14.842944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:14.847041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4e837-ba31-40b8-9814-20effe6a6b84 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:14.872671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:14.874214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:14.875382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:14.876325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] PATH_INFO: `/v3/projects/2e404e3752184b5c8bb8449f88db7573` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:14.880942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e404e3752184b5c8bb8449f88db7573)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:14.883939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:14.992608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f12c730fb242e9868838c412572e0e', 'request_id': 'req-41754e15-7bb5-4da4-b7c2-0412e4f3e928'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.005359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f12c730fb242e9868838c412572e0e', 'request_id': 'req-41754e15-7bb5-4da4-b7c2-0412e4f3e928'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.017535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e404e3752184b5c8bb8449f88db7573', 'request_id': 'req-057c0303-5273-499e-9758-27922ba4524c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.024988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f12c730fb242e9868838c412572e0e', 'request_id': 'req-41754e15-7bb5-4da4-b7c2-0412e4f3e928'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.033770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e404e3752184b5c8bb8449f88db7573', 'request_id': 'req-057c0303-5273-499e-9758-27922ba4524c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.048411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] The token cache is being invalidate because project b6f12c730fb242e9868838c412572e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:15.048981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.051966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.053303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e404e3752184b5c8bb8449f88db7573', 'request_id': 'req-057c0303-5273-499e-9758-27922ba4524c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.057905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 860/1724] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:14 2022] DELETE /identity/v3/projects/b6f12c730fb242e9868838c412572e0e => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:15.058848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:15.063701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:15.086610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] The token cache is being invalidate because project 2e404e3752184b5c8bb8449f88db7573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:15.087451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.090666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-057c0303-5273-499e-9758-27922ba4524c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.096952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 865/1725] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:14 2022] DELETE /identity/v3/projects/2e404e3752184b5c8bb8449f88db7573 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:15.099741 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad3b72-8a50-4952-bf1b-9d2439b80adc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:15.102347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad3b72-8a50-4952-bf1b-9d2439b80adc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:15.102678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad3b72-8a50-4952-bf1b-9d2439b80adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:15.194428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-87ad3b72-8a50-4952-bf1b-9d2439b80adc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:15.199211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-87ad3b72-8a50-4952-bf1b-9d2439b80adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:15.218498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41754e15-7bb5-4da4-b7c2-0412e4f3e928 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:15.307337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:15.310510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:15.369160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a0c6e-5b65-452d-9c8e-b2a7001d0564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:15.369886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a0c6e-5b65-452d-9c8e-b2a7001d0564 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:15.371339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a0c6e-5b65-452d-9c8e-b2a7001d0564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:15.374238 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a0c6e-5b65-452d-9c8e-b2a7001d0564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:15.380175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-192a0c6e-5b65-452d-9c8e-b2a7001d0564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:15.416230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 866/1726] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:15.417792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad3b72-8a50-4952-bf1b-9d2439b80adc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:15.419323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-87ad3b72-8a50-4952-bf1b-9d2439b80adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:15.454433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad3b72-8a50-4952-bf1b-9d2439b80adc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:15.468928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:15.469636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:15.469915 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:15.470189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] PATH_INFO: `/v3/users/8fbd19314824451bac1dce24dc31d38f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:15.474379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fbd19314824451bac1dce24dc31d38f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:15.476471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:15.611205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a0c6e-5b65-452d-9c8e-b2a7001d0564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:15.614386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a0c6e-5b65-452d-9c8e-b2a7001d0564 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:15.640760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 861/1727] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:15.644325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd4b3-115e-4fd5-9cc4-f773761c06ae None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:15.644624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd4b3-115e-4fd5-9cc4-f773761c06ae None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:15.645390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd4b3-115e-4fd5-9cc4-f773761c06ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:15.700490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fbd19314824451bac1dce24dc31d38f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.709091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-2a0cd4b3-115e-4fd5-9cc4-f773761c06ae None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:15.710323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-2a0cd4b3-115e-4fd5-9cc4-f773761c06ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:15.733883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fbd19314824451bac1dce24dc31d38f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.737145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 862/1728] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:15.740342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cd4b3-115e-4fd5-9cc4-f773761c06ae None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:15.770718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cd4b3-115e-4fd5-9cc4-f773761c06ae None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:15.795124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:15.797680 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fbd19314824451bac1dce24dc31d38f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.799909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fbd19314824451bac1dce24dc31d38f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.800935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fbd19314824451bac1dce24dc31d38f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:15.810512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:15.811563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:15.811882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:15.814935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:15.817657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:15.823843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:15.832630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 867/1729] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:15 2022] DELETE /identity/v3/users/8fbd19314824451bac1dce24dc31d38f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:15.859917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 863/1730] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:15.864271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:15.866438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:15.998895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:16.001099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:16.039621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856bd8f-d9fe-4b57-a6f0-d458f81b4deb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:16.055889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3513f8b-4d0c-47d3-94fd-9b2e7a56ed64 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:16.072492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:16.073490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fee5e-ca8a-4059-bc80-75f6b767232c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:16.073830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fee5e-ca8a-4059-bc80-75f6b767232c admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:16.074158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fee5e-ca8a-4059-bc80-75f6b767232c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:16.074424 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fee5e-ca8a-4059-bc80-75f6b767232c admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:16.075681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:16.075991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:16.077125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] PATH_INFO: `/v3/projects/fb10b19835d842da81152aab15fae61b` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:16.080186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fee5e-ca8a-4059-bc80-75f6b767232c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:16.080864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb10b19835d842da81152aab15fae61b)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:16.082191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fee5e-ca8a-4059-bc80-75f6b767232c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:16.084817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:16.171955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 868/1731] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:15 2022] POST /identity/v3/projects => generated 379 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:16.173832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fee5e-ca8a-4059-bc80-75f6b767232c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:16.251332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb10b19835d842da81152aab15fae61b', 'request_id': 'req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.266699 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb10b19835d842da81152aab15fae61b', 'request_id': 'req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.277827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:16.285675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb10b19835d842da81152aab15fae61b', 'request_id': 'req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.307505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] The token cache is being invalidate because project fb10b19835d842da81152aab15fae61b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:16.307814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.309168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.314444 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 864/1732] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:15 2022] DELETE /identity/v3/projects/fb10b19835d842da81152aab15fae61b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:16.318496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:16.320732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:16.362020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fee5e-ca8a-4059-bc80-75f6b767232c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:16.528673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:16.551553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86129edf-6ffc-4004-a6f3-e5a7392e4918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:16.554752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129edf-6ffc-4004-a6f3-e5a7392e4918 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:16.557318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129edf-6ffc-4004-a6f3-e5a7392e4918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:16.557663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129edf-6ffc-4004-a6f3-e5a7392e4918 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:16.558671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86129edf-6ffc-4004-a6f3-e5a7392e4918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:16.570123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86129edf-6ffc-4004-a6f3-e5a7392e4918 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:16.585714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9f4a5-9327-4ca0-8878-7a666edfbbc3 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:16.597977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:16.598824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:16.599256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:16.601668 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] PATH_INFO: `/v3/users/49e4e90c2c944fabb77bfe1e90d4f4ac` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:16.604919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e4e90c2c944fabb77bfe1e90d4f4ac)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:16.607209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:16.609078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 869/1733] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:16 2022] POST /identity/v3/users => generated 290 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:16.610650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5baf2-5baf-4f5f-aa73-e0326da192b1 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:16.610942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5baf2-5baf-4f5f-aa73-e0326da192b1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:16.611240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5baf2-5baf-4f5f-aa73-e0326da192b1 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:16.613001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 870/1734] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 10 10:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:03:16.614231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5baf2-5baf-4f5f-aa73-e0326da192b1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:16.617358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e7a5baf2-5baf-4f5f-aa73-e0326da192b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:16.672311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5baf2-5baf-4f5f-aa73-e0326da192b1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:16.685629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94871100-8a7b-440d-b109-38274e43387d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:16.687840 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871100-8a7b-440d-b109-38274e43387d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:16.688224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871100-8a7b-440d-b109-38274e43387d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:16.688601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871100-8a7b-440d-b109-38274e43387d admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:16.697014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94871100-8a7b-440d-b109-38274e43387d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:16.699968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94871100-8a7b-440d-b109-38274e43387d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:16.700513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e4e90c2c944fabb77bfe1e90d4f4ac'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.719362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e4e90c2c944fabb77bfe1e90d4f4ac'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.731320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 871/1735] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:16.734724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e4e90c2c944fabb77bfe1e90d4f4ac'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.736199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e782-fec1-48a4-9be6-8aa7d83918d1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:16.736619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e782-fec1-48a4-9be6-8aa7d83918d1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:16.739288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e782-fec1-48a4-9be6-8aa7d83918d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:16.759174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e4e90c2c944fabb77bfe1e90d4f4ac'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.760321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c082b9d2-a745-43c4-8e51-9abe334337d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e4e90c2c944fabb77bfe1e90d4f4ac'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:16.787725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8777e782-fec1-48a4-9be6-8aa7d83918d1 None None] MFA Rules not processed for user `ab9b7576a3834862af6abb7e0416f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:16.789329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 865/1736] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:16 2022] DELETE /identity/v3/users/49e4e90c2c944fabb77bfe1e90d4f4ac => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:16.790652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deeb467-694f-44c9-821f-036819dcd0b7 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:16.790938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deeb467-694f-44c9-821f-036819dcd0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:16.793353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deeb467-694f-44c9-821f-036819dcd0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:16.793770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8777e782-fec1-48a4-9be6-8aa7d83918d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:16.925343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 872/1737] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:03:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:16.926983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e782-fec1-48a4-9be6-8aa7d83918d1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:16.929846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8777e782-fec1-48a4-9be6-8aa7d83918d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:16.957452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0deeb467-694f-44c9-821f-036819dcd0b7 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:16.959019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0deeb467-694f-44c9-821f-036819dcd0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:17.130198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e782-fec1-48a4-9be6-8aa7d83918d1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:17.148820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 866/1738] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:03:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:17.154085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb0765-2a0a-446b-a2a7-25e53c336d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:17.155398 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb0765-2a0a-446b-a2a7-25e53c336d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:17.155751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb0765-2a0a-446b-a2a7-25e53c336d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:17.156194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb0765-2a0a-446b-a2a7-25e53c336d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:17.156965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb0765-2a0a-446b-a2a7-25e53c336d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:17.162458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb0765-2a0a-446b-a2a7-25e53c336d7f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:17.164732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deeb467-694f-44c9-821f-036819dcd0b7 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:17.167941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0deeb467-694f-44c9-821f-036819dcd0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:17.197327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deeb467-694f-44c9-821f-036819dcd0b7 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:17.205413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 873/1739] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:17.206014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2257ac-729b-421d-8f4e-2052b9ce725d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:17.206364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb0765-2a0a-446b-a2a7-25e53c336d7f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:17.206640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2257ac-729b-421d-8f4e-2052b9ce725d service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:17.206917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2257ac-729b-421d-8f4e-2052b9ce725d service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:17.207203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2257ac-729b-421d-8f4e-2052b9ce725d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:17.220900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:17.221952 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2257ac-729b-421d-8f4e-2052b9ce725d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:17.223990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2257ac-729b-421d-8f4e-2052b9ce725d service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:17.233613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb0765-2a0a-446b-a2a7-25e53c336d7f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:17.236263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 867/1740] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:17.248342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830355a5-36c4-4966-866a-d6cf264e5686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:17.249547 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830355a5-36c4-4966-866a-d6cf264e5686 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:17.250629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830355a5-36c4-4966-866a-d6cf264e5686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:17.251098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830355a5-36c4-4966-866a-d6cf264e5686 admin admin] PATH_INFO: `/v3/projects/6cdfbea73b3e457eb5415f5be77c2937/users/6963ee9577284a099a3120b1c03708ec/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:17.281292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830355a5-36c4-4966-866a-d6cf264e5686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cdfbea73b3e457eb5415f5be77c2937, user_id=6963ee9577284a099a3120b1c03708ec, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:17.284505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830355a5-36c4-4966-866a-d6cf264e5686 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:17.303267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 874/1741] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:17 2022] PUT /identity/v3/projects/6cdfbea73b3e457eb5415f5be77c2937/users/6963ee9577284a099a3120b1c03708ec/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:17.315715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ab4a9-13ae-465e-a8e5-fd371d3ba927 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:17.316978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ab4a9-13ae-465e-a8e5-fd371d3ba927 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:17.317458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ab4a9-13ae-465e-a8e5-fd371d3ba927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:17.344988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830355a5-36c4-4966-866a-d6cf264e5686 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:17.347372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-830355a5-36c4-4966-866a-d6cf264e5686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:17.387928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a62ab4a9-13ae-465e-a8e5-fd371d3ba927 None None] MFA Rules not processed for user `6963ee9577284a099a3120b1c03708ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:17.390663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a62ab4a9-13ae-465e-a8e5-fd371d3ba927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:17.428377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:17.575357 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830355a5-36c4-4966-866a-d6cf264e5686 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:17.589438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977768da-bc51-4351-b656-a2a9abf28c0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:17.590075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977768da-bc51-4351-b656-a2a9abf28c0d service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:17.590355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977768da-bc51-4351-b656-a2a9abf28c0d service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:17.590899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977768da-bc51-4351-b656-a2a9abf28c0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:17.594841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 868/1742] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:17.598932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ab4a9-13ae-465e-a8e5-fd371d3ba927 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:17.666812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977768da-bc51-4351-b656-a2a9abf28c0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:17.668476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977768da-bc51-4351-b656-a2a9abf28c0d service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:17.679665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 875/1743] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:03:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 10:03:17.697397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ab4a9-13ae-465e-a8e5-fd371d3ba927 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:17.703230 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b5c77-bc86-4ebc-b9d4-b8b6a1fcb068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:17.704124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b5c77-bc86-4ebc-b9d4-b8b6a1fcb068 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:17.704574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b5c77-bc86-4ebc-b9d4-b8b6a1fcb068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:17.705700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b5c77-bc86-4ebc-b9d4-b8b6a1fcb068 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:17.706518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b5c77-bc86-4ebc-b9d4-b8b6a1fcb068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:17.709983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b5c77-bc86-4ebc-b9d4-b8b6a1fcb068 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:17.821852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 869/1744] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:17 2022] POST /identity/v3/projects => generated 379 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:17.825184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977768da-bc51-4351-b656-a2a9abf28c0d service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:17.851295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977768da-bc51-4351-b656-a2a9abf28c0d service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:17.890790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b5c77-bc86-4ebc-b9d4-b8b6a1fcb068 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:17.892709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e93b5c77-bc86-4ebc-b9d4-b8b6a1fcb068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:18.078573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac3285-02ce-4198-ae48-4a837b06937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:18.080374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3285-02ce-4198-ae48-4a837b06937c admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:18.080374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3285-02ce-4198-ae48-4a837b06937c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:18.080374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3285-02ce-4198-ae48-4a837b06937c admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:18.080609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3285-02ce-4198-ae48-4a837b06937c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:18.096986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3285-02ce-4198-ae48-4a837b06937c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:18.117186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:18.130065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b5c77-bc86-4ebc-b9d4-b8b6a1fcb068 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:18.146234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:18.148134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:18.148134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:18.148134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] PATH_INFO: `/v3/projects/f7e4eb3676b64434894e2238e9635b66` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:18.151728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 876/1745] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:17 2022] POST /identity/v3/users => generated 296 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:18.152791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac3285-02ce-4198-ae48-4a837b06937c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:18.155313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e4eb3676b64434894e2238e9635b66)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:18.158583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:18.177435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac3285-02ce-4198-ae48-4a837b06937c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:18.193464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf52202-1c9a-4013-8754-99f670ef8d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:18.193464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf52202-1c9a-4013-8754-99f670ef8d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:18.193464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf52202-1c9a-4013-8754-99f670ef8d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:18.193464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf52202-1c9a-4013-8754-99f670ef8d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:18.194464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf52202-1c9a-4013-8754-99f670ef8d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:18.198826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf52202-1c9a-4013-8754-99f670ef8d9b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:18.233913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 877/1746] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:18.235014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf52202-1c9a-4013-8754-99f670ef8d9b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:18.263504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf52202-1c9a-4013-8754-99f670ef8d9b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:18.279941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:18.295014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e4eb3676b64434894e2238e9635b66', 'request_id': 'req-85826403-c83f-4da5-997d-97bfcd152958'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:18.314884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e4eb3676b64434894e2238e9635b66', 'request_id': 'req-85826403-c83f-4da5-997d-97bfcd152958'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:18.346549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e4eb3676b64434894e2238e9635b66', 'request_id': 'req-85826403-c83f-4da5-997d-97bfcd152958'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:18.400544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] The token cache is being invalidate because project f7e4eb3676b64434894e2238e9635b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:18.400544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:18.400544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-85826403-c83f-4da5-997d-97bfcd152958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:18.413527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 870/1747] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:17 2022] DELETE /identity/v3/projects/f7e4eb3676b64434894e2238e9635b66 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:18.414760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51294971-6d4a-4fef-bc43-9282727c1eea None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:18.415099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51294971-6d4a-4fef-bc43-9282727c1eea None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:18.415378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51294971-6d4a-4fef-bc43-9282727c1eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:18.457581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-51294971-6d4a-4fef-bc43-9282727c1eea None None] MFA Rules not processed for user `2dd3653b931643318f1a1b0820620277`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:18.458863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-51294971-6d4a-4fef-bc43-9282727c1eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:18.518659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a95f5-b94a-48e6-b8d3-90a9dc9eb995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:18.520625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a95f5-b94a-48e6-b8d3-90a9dc9eb995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:18.524098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a95f5-b94a-48e6-b8d3-90a9dc9eb995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:18.524098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a95f5-b94a-48e6-b8d3-90a9dc9eb995 admin admin] PATH_INFO: `/v3/projects/3690ab2d887e42dca6d5124357e55131/users/8af45ee4682448f4b4b7915b47cc2c13/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:18.628604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a95f5-b94a-48e6-b8d3-90a9dc9eb995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3690ab2d887e42dca6d5124357e55131, user_id=8af45ee4682448f4b4b7915b47cc2c13, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:18.645257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a95f5-b94a-48e6-b8d3-90a9dc9eb995 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:18.675848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 871/1748] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:18.683346 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51294971-6d4a-4fef-bc43-9282727c1eea None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:18.694719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-51294971-6d4a-4fef-bc43-9282727c1eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:18.708145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 878/1749] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:18 2022] PUT /identity/v3/projects/3690ab2d887e42dca6d5124357e55131/users/8af45ee4682448f4b4b7915b47cc2c13/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:18.710148 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a95f5-b94a-48e6-b8d3-90a9dc9eb995 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:18.714301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-413a95f5-b94a-48e6-b8d3-90a9dc9eb995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:18.778381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51294971-6d4a-4fef-bc43-9282727c1eea None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:19.031805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a95f5-b94a-48e6-b8d3-90a9dc9eb995 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:19.044093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:19.044892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:19.045185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:19.045574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] PATH_INFO: `/v3/projects/e7261336c06443029fc9c880fc86f80e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:19.047253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7261336c06443029fc9c880fc86f80e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:19.049195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:19.051511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863392b-667f-4512-a88f-53978425064f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:19.052169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863392b-667f-4512-a88f-53978425064f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:19.052469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863392b-667f-4512-a88f-53978425064f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:19.052798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863392b-667f-4512-a88f-53978425064f admin admin] PATH_INFO: `/v3/users/2fa32f52833d4f5498238a56a5c4a8c6` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:19.058251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863392b-667f-4512-a88f-53978425064f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fa32f52833d4f5498238a56a5c4a8c6)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:19.062459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863392b-667f-4512-a88f-53978425064f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:19.214595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4863392b-667f-4512-a88f-53978425064f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fa32f52833d4f5498238a56a5c4a8c6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.222890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7261336c06443029fc9c880fc86f80e', 'request_id': 'req-538549c2-61e0-4375-8f79-de087601f0d5'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.238276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4863392b-667f-4512-a88f-53978425064f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fa32f52833d4f5498238a56a5c4a8c6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.244175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7261336c06443029fc9c880fc86f80e', 'request_id': 'req-538549c2-61e0-4375-8f79-de087601f0d5'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.258102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:19.275125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7261336c06443029fc9c880fc86f80e', 'request_id': 'req-538549c2-61e0-4375-8f79-de087601f0d5'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.305908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4863392b-667f-4512-a88f-53978425064f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fa32f52833d4f5498238a56a5c4a8c6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.311810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4863392b-667f-4512-a88f-53978425064f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fa32f52833d4f5498238a56a5c4a8c6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.316809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4863392b-667f-4512-a88f-53978425064f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fa32f52833d4f5498238a56a5c4a8c6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.351179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] The token cache is being invalidate because project e7261336c06443029fc9c880fc86f80e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:19.351490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.356955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.374362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 872/1750] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:18 2022] DELETE /identity/v3/projects/e7261336c06443029fc9c880fc86f80e => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:19.379567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:19.383109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 879/1751] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:18 2022] DELETE /identity/v3/users/2fa32f52833d4f5498238a56a5c4a8c6 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:19.384582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:19.385790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863392b-667f-4512-a88f-53978425064f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:19.393157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4863392b-667f-4512-a88f-53978425064f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:19.460784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:19.726904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538549c2-61e0-4375-8f79-de087601f0d5 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:19.754035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:19.754815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:19.755949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863392b-667f-4512-a88f-53978425064f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:19.758035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:19.758347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] PATH_INFO: `/v3/projects/b10437222ace4a8091f5cc7c49898339` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:19.762888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10437222ace4a8091f5cc7c49898339)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:19.767279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:19.774262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e95816-d416-4151-aba8-58719878da97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:19.774897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e95816-d416-4151-aba8-58719878da97 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:19.775211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e95816-d416-4151-aba8-58719878da97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:19.779642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e95816-d416-4151-aba8-58719878da97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:19.780456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e95816-d416-4151-aba8-58719878da97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:19.787446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e95816-d416-4151-aba8-58719878da97 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:19.845067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 880/1752] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:19.849310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bb7bc-1579-45d1-b6a7-5606af9c3568 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:19.851827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bb7bc-1579-45d1-b6a7-5606af9c3568 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:19.852422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bb7bc-1579-45d1-b6a7-5606af9c3568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:19.901646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-928bb7bc-1579-45d1-b6a7-5606af9c3568 None None] MFA Rules not processed for user `befbca3738f0481296b63c70d6d2a797`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:19.908491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-928bb7bc-1579-45d1-b6a7-5606af9c3568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:19.961240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10437222ace4a8091f5cc7c49898339', 'request_id': 'req-7851f934-354c-48c6-9979-988d0813ca2f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.976979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10437222ace4a8091f5cc7c49898339', 'request_id': 'req-7851f934-354c-48c6-9979-988d0813ca2f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:19.999877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10437222ace4a8091f5cc7c49898339', 'request_id': 'req-7851f934-354c-48c6-9979-988d0813ca2f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:20.026709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] The token cache is being invalidate because project b10437222ace4a8091f5cc7c49898339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:20.027411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:20.029620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7851f934-354c-48c6-9979-988d0813ca2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:20.039310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 873/1753] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:19 2022] DELETE /identity/v3/projects/b10437222ace4a8091f5cc7c49898339 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:20.043199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2f0d0-4100-4fd5-977b-a52790c9fa74 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:20.043545 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2f0d0-4100-4fd5-977b-a52790c9fa74 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:20.043922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2f0d0-4100-4fd5-977b-a52790c9fa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:20.098798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-7ba2f0d0-4100-4fd5-977b-a52790c9fa74 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:20.102255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7ba2f0d0-4100-4fd5-977b-a52790c9fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:20.133349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 881/1754] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:20.133349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bb7bc-1579-45d1-b6a7-5606af9c3568 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:20.136673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-928bb7bc-1579-45d1-b6a7-5606af9c3568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:20.202698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bb7bc-1579-45d1-b6a7-5606af9c3568 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:20.294693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 874/1755] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:20.296773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7c2d7-a03e-430b-9156-c96e1f8ac900 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:20.297056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7c2d7-a03e-430b-9156-c96e1f8ac900 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:20.297338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7c2d7-a03e-430b-9156-c96e1f8ac900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:20.345354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-4eb7c2d7-a03e-430b-9156-c96e1f8ac900 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:20.348370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4eb7c2d7-a03e-430b-9156-c96e1f8ac900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:20.402225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 875/1756] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:20.403841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aff7fc-ff01-4901-ab54-73669cca72b3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:20.404126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aff7fc-ff01-4901-ab54-73669cca72b3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:20.404969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aff7fc-ff01-4901-ab54-73669cca72b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:20.469986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29818c-c341-4d5b-a8bb-d7985b687baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:20.470692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29818c-c341-4d5b-a8bb-d7985b687baf admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:20.471037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29818c-c341-4d5b-a8bb-d7985b687baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:20.471266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29818c-c341-4d5b-a8bb-d7985b687baf admin admin] PATH_INFO: `/v3/projects/3690ab2d887e42dca6d5124357e55131/users/8af45ee4682448f4b4b7915b47cc2c13/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:20.475739 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-58aff7fc-ff01-4901-ab54-73669cca72b3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:20.482926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29818c-c341-4d5b-a8bb-d7985b687baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3690ab2d887e42dca6d5124357e55131, user_id=8af45ee4682448f4b4b7915b47cc2c13, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:20.483492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-58aff7fc-ff01-4901-ab54-73669cca72b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:20.506033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29818c-c341-4d5b-a8bb-d7985b687baf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:20.542451 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 876/1757] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:20.544117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f3890-d9c5-405d-ad4b-9a29dddd8eff None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:20.544563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f3890-d9c5-405d-ad4b-9a29dddd8eff None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:20.545019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f3890-d9c5-405d-ad4b-9a29dddd8eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:20.555448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 882/1758] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:20 2022] PUT /identity/v3/projects/3690ab2d887e42dca6d5124357e55131/users/8af45ee4682448f4b4b7915b47cc2c13/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:20.570692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29818c-c341-4d5b-a8bb-d7985b687baf admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:20.585599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a30f3890-d9c5-405d-ad4b-9a29dddd8eff None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:20.587810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a30f3890-d9c5-405d-ad4b-9a29dddd8eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:20.633416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:20.679043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:20.765703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29818c-c341-4d5b-a8bb-d7985b687baf admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:20.779660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08180070-ba14-4da0-b5d1-5ecfe43f43b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:20.780298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08180070-ba14-4da0-b5d1-5ecfe43f43b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:20.780577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08180070-ba14-4da0-b5d1-5ecfe43f43b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:20.780849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08180070-ba14-4da0-b5d1-5ecfe43f43b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:20.781516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08180070-ba14-4da0-b5d1-5ecfe43f43b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:20.782619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08180070-ba14-4da0-b5d1-5ecfe43f43b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:20.784280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08180070-ba14-4da0-b5d1-5ecfe43f43b2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:20.788229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 877/1759] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:20.790146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead797fb-b0be-4782-8e9b-00b81fc95d1d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:20.790436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead797fb-b0be-4782-8e9b-00b81fc95d1d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:20.790709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead797fb-b0be-4782-8e9b-00b81fc95d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:20.855987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 883/1760] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:20.863883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08180070-ba14-4da0-b5d1-5ecfe43f43b2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:20.899133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08180070-ba14-4da0-b5d1-5ecfe43f43b2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:20.919733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6914d-b8d5-4509-8e22-ccdafade57a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:20.921321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6914d-b8d5-4509-8e22-ccdafade57a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:20.922001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6914d-b8d5-4509-8e22-ccdafade57a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:20.922606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6914d-b8d5-4509-8e22-ccdafade57a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:20.923535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6914d-b8d5-4509-8e22-ccdafade57a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:20.924157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6914d-b8d5-4509-8e22-ccdafade57a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:20.927414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ead797fb-b0be-4782-8e9b-00b81fc95d1d None None] MFA Rules not processed for user `8af45ee4682448f4b4b7915b47cc2c13`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:20.928492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6914d-b8d5-4509-8e22-ccdafade57a8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:20.929272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ead797fb-b0be-4782-8e9b-00b81fc95d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:20.958171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 884/1761] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:20.959333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6914d-b8d5-4509-8e22-ccdafade57a8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:20.980732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6914d-b8d5-4509-8e22-ccdafade57a8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:20.996207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d848235-c8ff-4424-a7a8-ca4f3df4e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:20.997032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d848235-c8ff-4424-a7a8-ca4f3df4e51c admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:20.997467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d848235-c8ff-4424-a7a8-ca4f3df4e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:20.997844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d848235-c8ff-4424-a7a8-ca4f3df4e51c admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:20.999164 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d848235-c8ff-4424-a7a8-ca4f3df4e51c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:21.003126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d848235-c8ff-4424-a7a8-ca4f3df4e51c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:21.079834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 885/1762] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:20 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:21.080900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d848235-c8ff-4424-a7a8-ca4f3df4e51c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:21.176004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 878/1763] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:21.176870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead797fb-b0be-4782-8e9b-00b81fc95d1d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:21.199387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead797fb-b0be-4782-8e9b-00b81fc95d1d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:21.255492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d848235-c8ff-4424-a7a8-ca4f3df4e51c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:21.273762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a4d8b-718b-4231-af8c-db2a83d0aac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:21.275059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a4d8b-718b-4231-af8c-db2a83d0aac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:21.276811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a4d8b-718b-4231-af8c-db2a83d0aac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:21.278122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a4d8b-718b-4231-af8c-db2a83d0aac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:21.282150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a4d8b-718b-4231-af8c-db2a83d0aac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:21.284509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a4d8b-718b-4231-af8c-db2a83d0aac4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:21.356960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 886/1764] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:21 2022] POST /identity/v3/projects => generated 407 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:21.360852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a4d8b-718b-4231-af8c-db2a83d0aac4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:21.363857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a96a4d8b-718b-4231-af8c-db2a83d0aac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:21.406278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585abe74-bfc6-4ace-a403-5f1a436f1a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:21.406994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585abe74-bfc6-4ace-a403-5f1a436f1a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:21.407292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585abe74-bfc6-4ace-a403-5f1a436f1a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:21.407633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585abe74-bfc6-4ace-a403-5f1a436f1a36 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:21.408214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585abe74-bfc6-4ace-a403-5f1a436f1a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:21.425905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585abe74-bfc6-4ace-a403-5f1a436f1a36 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:21.524455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 879/1765] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:21 2022] POST /identity/v3/users => generated 286 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:21.528225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585abe74-bfc6-4ace-a403-5f1a436f1a36 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:21.579264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585abe74-bfc6-4ace-a403-5f1a436f1a36 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:21.654245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a4d8b-718b-4231-af8c-db2a83d0aac4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:21.667829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366e4e8-33d9-4ef2-9b74-9cd5427d0809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:21.668474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e4e8-33d9-4ef2-9b74-9cd5427d0809 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:21.670135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e4e8-33d9-4ef2-9b74-9cd5427d0809 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:21.671981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e4e8-33d9-4ef2-9b74-9cd5427d0809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:21.729285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:21.772152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:21.872065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceea6c-8a0c-4d3b-8641-60b6841589ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:21.873094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceea6c-8a0c-4d3b-8641-60b6841589ef admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:21.873094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceea6c-8a0c-4d3b-8641-60b6841589ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:21.881828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceea6c-8a0c-4d3b-8641-60b6841589ef admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:21.883912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceea6c-8a0c-4d3b-8641-60b6841589ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:21.898514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceea6c-8a0c-4d3b-8641-60b6841589ef admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:21.971688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 880/1766] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:21 2022] POST /identity/v3/users => generated 304 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:21.979519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceea6c-8a0c-4d3b-8641-60b6841589ef admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:21.980314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366e4e8-33d9-4ef2-9b74-9cd5427d0809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:21.981778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366e4e8-33d9-4ef2-9b74-9cd5427d0809 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:22.018894 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceea6c-8a0c-4d3b-8641-60b6841589ef admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:22.021739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 887/1767] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:22.040766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366e4e8-33d9-4ef2-9b74-9cd5427d0809 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:22.056300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4272da2-2a17-4ec9-ba62-9d804b1ca06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:22.062353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4272da2-2a17-4ec9-ba62-9d804b1ca06b admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:22.062643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4272da2-2a17-4ec9-ba62-9d804b1ca06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:22.062915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4272da2-2a17-4ec9-ba62-9d804b1ca06b admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:22.063602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4272da2-2a17-4ec9-ba62-9d804b1ca06b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:22.067944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4272da2-2a17-4ec9-ba62-9d804b1ca06b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:22.089228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366e4e8-33d9-4ef2-9b74-9cd5427d0809 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:22.114426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718179d-fc50-4220-8100-e9354b7991d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:22.114426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718179d-fc50-4220-8100-e9354b7991d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:22.114426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718179d-fc50-4220-8100-e9354b7991d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:22.114764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718179d-fc50-4220-8100-e9354b7991d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:22.115264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718179d-fc50-4220-8100-e9354b7991d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:22.116861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 881/1768] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:22.117789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4272da2-2a17-4ec9-ba62-9d804b1ca06b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:22.118616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718179d-fc50-4220-8100-e9354b7991d1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:22.165850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 888/1769] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:22.166730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718179d-fc50-4220-8100-e9354b7991d1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:22.169465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4272da2-2a17-4ec9-ba62-9d804b1ca06b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:22.181171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed34439-feb8-42e0-9877-0fdb028d446b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:22.182156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed34439-feb8-42e0-9877-0fdb028d446b admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:22.182426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed34439-feb8-42e0-9877-0fdb028d446b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:22.182696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed34439-feb8-42e0-9877-0fdb028d446b admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:22.188443 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed34439-feb8-42e0-9877-0fdb028d446b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:22.192619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed34439-feb8-42e0-9877-0fdb028d446b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:22.193579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718179d-fc50-4220-8100-e9354b7991d1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:22.205800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8b093-86fe-4ae6-a9ea-4eeb123f59c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:22.206384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8b093-86fe-4ae6-a9ea-4eeb123f59c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:22.206636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8b093-86fe-4ae6-a9ea-4eeb123f59c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:22.207003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8b093-86fe-4ae6-a9ea-4eeb123f59c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:22.211512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8b093-86fe-4ae6-a9ea-4eeb123f59c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:22.215719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8b093-86fe-4ae6-a9ea-4eeb123f59c3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:22.222520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 882/1770] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:22.227356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed34439-feb8-42e0-9877-0fdb028d446b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:22.251252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 889/1771] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:22.257506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8b093-86fe-4ae6-a9ea-4eeb123f59c3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:22.289678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed34439-feb8-42e0-9877-0fdb028d446b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:22.308368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8b093-86fe-4ae6-a9ea-4eeb123f59c3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:22.317256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca288102-b4a1-436b-9993-116a4d779626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:22.318386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:22.319274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca288102-b4a1-436b-9993-116a4d779626 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:22.319539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:22.320000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:22.321143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca288102-b4a1-436b-9993-116a4d779626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:22.321412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:22.321636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca288102-b4a1-436b-9993-116a4d779626 admin admin] PATH_INFO: `/v3/projects/37d0c4f10bec4455828e2fbe3ac526ca/users/3267b05f29b840c5bd42d446117d8b16/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:22.325366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:22.396370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca288102-b4a1-436b-9993-116a4d779626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d0c4f10bec4455828e2fbe3ac526ca, user_id=3267b05f29b840c5bd42d446117d8b16, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:22.410855 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca288102-b4a1-436b-9993-116a4d779626 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:22.455243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 883/1772] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:22 2022] PUT /identity/v3/projects/37d0c4f10bec4455828e2fbe3ac526ca/users/3267b05f29b840c5bd42d446117d8b16/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:22.457249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca288102-b4a1-436b-9993-116a4d779626 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:22.608303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:22.617212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:22.648039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca288102-b4a1-436b-9993-116a4d779626 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:22.668302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b2dd3-c19d-47f1-96ab-5e70b5019ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:22.668982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b2dd3-c19d-47f1-96ab-5e70b5019ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:22.669256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b2dd3-c19d-47f1-96ab-5e70b5019ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:22.669526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b2dd3-c19d-47f1-96ab-5e70b5019ae2 admin admin] PATH_INFO: `/v3/projects/c0e8e63b98e144fb98f04ceea3180cf5/users/f05a43407a8d487d8d2c145ac0ecbca9/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:22.678492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 890/1773] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:22.682341 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:22.686988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:22.744570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b2dd3-c19d-47f1-96ab-5e70b5019ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0e8e63b98e144fb98f04ceea3180cf5, user_id=f05a43407a8d487d8d2c145ac0ecbca9, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:22.749783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b2dd3-c19d-47f1-96ab-5e70b5019ae2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:22.772194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e1328-cf07-486e-9727-22219dd68524 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:22.776758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 884/1774] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:22 2022] PUT /identity/v3/projects/c0e8e63b98e144fb98f04ceea3180cf5/users/f05a43407a8d487d8d2c145ac0ecbca9/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:22.787424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0573ee2-900a-4880-a1b4-9f1a6945d68f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:22.787734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0573ee2-900a-4880-a1b4-9f1a6945d68f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:22.791538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0573ee2-900a-4880-a1b4-9f1a6945d68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:22.869289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d0573ee2-900a-4880-a1b4-9f1a6945d68f None None] MFA Rules not processed for user `3267b05f29b840c5bd42d446117d8b16`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:22.871219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d0573ee2-900a-4880-a1b4-9f1a6945d68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:23.082413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:23.108206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 885/1775] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:23.112056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e571ad-9db4-4c20-b7a5-b70edf63b6db None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:23.112932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e571ad-9db4-4c20-b7a5-b70edf63b6db None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:23.114252 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e571ad-9db4-4c20-b7a5-b70edf63b6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:23.119048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:23.120222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:23.120662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:23.127656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] PATH_INFO: `/v3/projects/ea21ae02cdc749b8aac85786da26260f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:23.130280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea21ae02cdc749b8aac85786da26260f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:23.140995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:23.204834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-47e571ad-9db4-4c20-b7a5-b70edf63b6db None None] MFA Rules not processed for user `ab9b7576a3834862af6abb7e0416f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:23.211744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-47e571ad-9db4-4c20-b7a5-b70edf63b6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:23.270792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:23.356788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea21ae02cdc749b8aac85786da26260f', 'request_id': 'req-8e27999d-f05b-4fe4-8012-971d5ac75d28'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:23.413233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea21ae02cdc749b8aac85786da26260f', 'request_id': 'req-8e27999d-f05b-4fe4-8012-971d5ac75d28'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:23.432002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 886/1776] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 10 10:03:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:23.433297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea21ae02cdc749b8aac85786da26260f', 'request_id': 'req-8e27999d-f05b-4fe4-8012-971d5ac75d28'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:23.437106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea76a4-db81-42a1-8765-e24919656a54 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:23.437733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea76a4-db81-42a1-8765-e24919656a54 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:23.438511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea76a4-db81-42a1-8765-e24919656a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:23.470946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] The token cache is being invalidate because project ea21ae02cdc749b8aac85786da26260f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:23.481043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:23.481947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:23.490890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 891/1777] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:22 2022] DELETE /identity/v3/projects/ea21ae02cdc749b8aac85786da26260f => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:23.490890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:23.493259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:23.509317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-7eea76a4-db81-42a1-8765-e24919656a54 None None] MFA Rules not processed for user `f05a43407a8d487d8d2c145ac0ecbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:23.513086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7eea76a4-db81-42a1-8765-e24919656a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:23.680549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27999d-f05b-4fe4-8012-971d5ac75d28 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:23.695378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf0b1f-5bf9-46b5-8ec9-0b6126a172b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:23.696957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf0b1f-5bf9-46b5-8ec9-0b6126a172b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:23.697539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf0b1f-5bf9-46b5-8ec9-0b6126a172b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:23.698712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf0b1f-5bf9-46b5-8ec9-0b6126a172b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:23.699500 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf0b1f-5bf9-46b5-8ec9-0b6126a172b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:23.707003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf0b1f-5bf9-46b5-8ec9-0b6126a172b9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:23.757285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 887/1778] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:23 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:23.760856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb00a2a-527e-40ff-9163-46e00b278208 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:23.761824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb00a2a-527e-40ff-9163-46e00b278208 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:23.764098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb00a2a-527e-40ff-9163-46e00b278208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:23.791094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 892/1779] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:23 2022] POST /identity/v3/projects => generated 373 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:23.792121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf0b1f-5bf9-46b5-8ec9-0b6126a172b9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:23.798248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-badf0b1f-5bf9-46b5-8ec9-0b6126a172b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:23.813666 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ebb00a2a-527e-40ff-9163-46e00b278208 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:23.817733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ebb00a2a-527e-40ff-9163-46e00b278208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:23.951583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf0b1f-5bf9-46b5-8ec9-0b6126a172b9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:23.965966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:23.966559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:23.966878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:23.968249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:23.979648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:24.000442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:24.002200 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:24.023451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 893/1780] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:24.027089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:24.033956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:24.049417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 888/1781] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:24.050387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb00a2a-527e-40ff-9163-46e00b278208 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:24.080762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22129b10-ad0a-4349-a2dc-959423c6967d service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:24.081644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27f4dc-7233-4751-b9ff-8533d697cb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:24.082447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27f4dc-7233-4751-b9ff-8533d697cb50 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:24.082777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27f4dc-7233-4751-b9ff-8533d697cb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:24.083138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27f4dc-7233-4751-b9ff-8533d697cb50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:24.083844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27f4dc-7233-4751-b9ff-8533d697cb50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:24.088047 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27f4dc-7233-4751-b9ff-8533d697cb50 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:24.094472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb00a2a-527e-40ff-9163-46e00b278208 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:24.108156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbef07-5774-44c1-9faa-ab82d806c0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:24.108754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbef07-5774-44c1-9faa-ab82d806c0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:24.109029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbef07-5774-44c1-9faa-ab82d806c0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:24.109297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbef07-5774-44c1-9faa-ab82d806c0e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:24.112513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbef07-5774-44c1-9faa-ab82d806c0e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:24.117008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbef07-5774-44c1-9faa-ab82d806c0e7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:24.158196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 894/1782] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:24 2022] POST /identity/v3/projects => generated 405 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:24.158196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27f4dc-7233-4751-b9ff-8533d697cb50 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:24.161000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ce27f4dc-7233-4751-b9ff-8533d697cb50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:24.189526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 889/1783] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:24 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:24.191175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee1309-9bee-4735-ba1a-4a730fbf1bfa None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:24.191748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee1309-9bee-4735-ba1a-4a730fbf1bfa None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:24.194729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee1309-9bee-4735-ba1a-4a730fbf1bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:24.250088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e9ee1309-9bee-4735-ba1a-4a730fbf1bfa None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:24.258258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e9ee1309-9bee-4735-ba1a-4a730fbf1bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:24.297738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:24.302669 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:24.468555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 890/1784] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:24.470595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee1309-9bee-4735-ba1a-4a730fbf1bfa None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:24.506241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee1309-9bee-4735-ba1a-4a730fbf1bfa None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:24.521069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3458ce-41dd-4408-b8a8-dd7884b65dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:24.522100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3458ce-41dd-4408-b8a8-dd7884b65dad admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:24.522447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3458ce-41dd-4408-b8a8-dd7884b65dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:24.523353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3458ce-41dd-4408-b8a8-dd7884b65dad admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:24.526027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3458ce-41dd-4408-b8a8-dd7884b65dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:24.531529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3458ce-41dd-4408-b8a8-dd7884b65dad admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:24.587069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27f4dc-7233-4751-b9ff-8533d697cb50 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:24.618738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 891/1785] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:24 2022] POST /identity/v3/users => generated 309 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:24.619726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3458ce-41dd-4408-b8a8-dd7884b65dad admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:24.622679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:24.623424 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:24.623703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:24.623976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] PATH_INFO: `/v3/users/60046731fbb14ee1897a2c4fddf52b92` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:24.630505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60046731fbb14ee1897a2c4fddf52b92)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:24.639618 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:24.683069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3458ce-41dd-4408-b8a8-dd7884b65dad admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:24.703827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b1e91-d653-4020-94bb-bfeda7e26237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:24.706583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b1e91-d653-4020-94bb-bfeda7e26237 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:24.706870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b1e91-d653-4020-94bb-bfeda7e26237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:24.707172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b1e91-d653-4020-94bb-bfeda7e26237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:24.707845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b1e91-d653-4020-94bb-bfeda7e26237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:24.720420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b1e91-d653-4020-94bb-bfeda7e26237 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:24.768098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 892/1786] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:24.772207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b1e91-d653-4020-94bb-bfeda7e26237 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:24.803169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60046731fbb14ee1897a2c4fddf52b92'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:24.816375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b1e91-d653-4020-94bb-bfeda7e26237 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:24.829538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60046731fbb14ee1897a2c4fddf52b92'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:24.837830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:24.840667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:24.840957 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:24.841230 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:24.843434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:24.843785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:24.849584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:24.863853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60046731fbb14ee1897a2c4fddf52b92'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:24.864672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60046731fbb14ee1897a2c4fddf52b92'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:24.877096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60046731fbb14ee1897a2c4fddf52b92'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:24.904743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 893/1787] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:24.908500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:24.911496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:24.917948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 895/1788] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:24 2022] DELETE /identity/v3/users/60046731fbb14ee1897a2c4fddf52b92 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:24.924575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:24.931769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:24.934666 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff34eb-6c47-4a2a-81f7-9ceb81eb63e1 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:25.092588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c48a3-c2a9-4f48-a452-2dc43a799344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:25.092588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c48a3-c2a9-4f48-a452-2dc43a799344 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:25.092588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c48a3-c2a9-4f48-a452-2dc43a799344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:25.093049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c48a3-c2a9-4f48-a452-2dc43a799344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:25.094157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c48a3-c2a9-4f48-a452-2dc43a799344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:25.098320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c48a3-c2a9-4f48-a452-2dc43a799344 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:25.144034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 894/1789] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:25.145021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c48a3-c2a9-4f48-a452-2dc43a799344 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:25.155165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3a8c48a3-c2a9-4f48-a452-2dc43a799344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:25.173588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ddd76-df51-4094-83c6-52773e0c85d8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:25.192675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f15a14-ff4a-4fab-a7e9-09bfc91eecc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:25.193397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f15a14-ff4a-4fab-a7e9-09bfc91eecc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:25.193683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f15a14-ff4a-4fab-a7e9-09bfc91eecc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:25.193957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f15a14-ff4a-4fab-a7e9-09bfc91eecc4 admin admin] PATH_INFO: `/v3/projects/27eaea7979df4282b78601f3ac0cd441/users/df1b9e2b30694eb0a30e53f12906f016/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:25.197173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c48a3-c2a9-4f48-a452-2dc43a799344 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:25.225814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280baca-60fc-45bb-9de2-63dca04595a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:25.225814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280baca-60fc-45bb-9de2-63dca04595a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:25.225814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280baca-60fc-45bb-9de2-63dca04595a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:25.225814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280baca-60fc-45bb-9de2-63dca04595a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:25.226558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280baca-60fc-45bb-9de2-63dca04595a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:25.238629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280baca-60fc-45bb-9de2-63dca04595a3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:25.257651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f15a14-ff4a-4fab-a7e9-09bfc91eecc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27eaea7979df4282b78601f3ac0cd441, user_id=df1b9e2b30694eb0a30e53f12906f016, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:25.262377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f15a14-ff4a-4fab-a7e9-09bfc91eecc4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:25.340093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 896/1790] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:24 2022] PUT /identity/v3/projects/27eaea7979df4282b78601f3ac0cd441/users/df1b9e2b30694eb0a30e53f12906f016/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:25.352363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dc41c-3667-463b-91e9-ed03afe20b64 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:25.352636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dc41c-3667-463b-91e9-ed03afe20b64 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:25.352916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dc41c-3667-463b-91e9-ed03afe20b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:25.458056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 895/1791] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:25 2022] POST /identity/v3/projects => generated 377 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:25.459152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280baca-60fc-45bb-9de2-63dca04595a3 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:25.522906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-832dc41c-3667-463b-91e9-ed03afe20b64 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:25.533501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-832dc41c-3667-463b-91e9-ed03afe20b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:25.656499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:25.815245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:25.854745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280baca-60fc-45bb-9de2-63dca04595a3 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:25.866885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1e94-1515-414d-9bc9-5a5c5cad51fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:25.867637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1e94-1515-414d-9bc9-5a5c5cad51fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:25.867912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1e94-1515-414d-9bc9-5a5c5cad51fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:25.868189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1e94-1515-414d-9bc9-5a5c5cad51fb admin admin] PATH_INFO: `/v3/projects/7765f0bee2b2436383c1805f87af3e33/users/9ac3492e69ae418b8d9a161f6472e340/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:25.905525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1e94-1515-414d-9bc9-5a5c5cad51fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7765f0bee2b2436383c1805f87af3e33, user_id=9ac3492e69ae418b8d9a161f6472e340, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:25.916246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 897/1792] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 576 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:25.917127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1e94-1515-414d-9bc9-5a5c5cad51fb admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:25.917922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dc41c-3667-463b-91e9-ed03afe20b64 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:25.952656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dc41c-3667-463b-91e9-ed03afe20b64 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:25.956962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 896/1793] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:25 2022] PUT /identity/v3/projects/7765f0bee2b2436383c1805f87af3e33/users/9ac3492e69ae418b8d9a161f6472e340/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:25.958527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1e94-1515-414d-9bc9-5a5c5cad51fb admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:26.171543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1e94-1515-414d-9bc9-5a5c5cad51fb admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:26.185381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007822f1-0480-419a-8336-d6ac1031638d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:26.186591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007822f1-0480-419a-8336-d6ac1031638d admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:26.186870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007822f1-0480-419a-8336-d6ac1031638d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:26.187219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007822f1-0480-419a-8336-d6ac1031638d admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:26.187937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007822f1-0480-419a-8336-d6ac1031638d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:26.192474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007822f1-0480-419a-8336-d6ac1031638d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:26.199887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32035a2-dc03-442d-ad92-0c6361416403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:26.200530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32035a2-dc03-442d-ad92-0c6361416403 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:26.200819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32035a2-dc03-442d-ad92-0c6361416403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:26.201505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32035a2-dc03-442d-ad92-0c6361416403 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:26.202172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32035a2-dc03-442d-ad92-0c6361416403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:26.204154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32035a2-dc03-442d-ad92-0c6361416403 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:26.247715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 898/1794] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:26.249710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32035a2-dc03-442d-ad92-0c6361416403 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:26.266361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 897/1795] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:25 2022] POST /identity/v3/users => generated 289 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:26.272104 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007822f1-0480-419a-8336-d6ac1031638d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:26.280062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-007822f1-0480-419a-8336-d6ac1031638d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:26.303281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32035a2-dc03-442d-ad92-0c6361416403 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:26.326695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56bc3-4072-4f45-aa91-815398df50f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:26.327535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56bc3-4072-4f45-aa91-815398df50f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:26.327824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56bc3-4072-4f45-aa91-815398df50f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:26.328105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56bc3-4072-4f45-aa91-815398df50f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:26.328775 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db56bc3-4072-4f45-aa91-815398df50f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:26.330394 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db56bc3-4072-4f45-aa91-815398df50f1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:26.375678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 899/1796] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:26.396847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56bc3-4072-4f45-aa91-815398df50f1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:26.408868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56bc3-4072-4f45-aa91-815398df50f1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:26.437699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975711d0-a412-449b-8f0a-cc4d406b10a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:26.438913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975711d0-a412-449b-8f0a-cc4d406b10a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:26.439254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975711d0-a412-449b-8f0a-cc4d406b10a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:26.439532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975711d0-a412-449b-8f0a-cc4d406b10a7 admin admin] PATH_INFO: `/v3/projects/27eaea7979df4282b78601f3ac0cd441/users/df1b9e2b30694eb0a30e53f12906f016/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:26.449282 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975711d0-a412-449b-8f0a-cc4d406b10a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27eaea7979df4282b78601f3ac0cd441, user_id=df1b9e2b30694eb0a30e53f12906f016, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:26.464274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975711d0-a412-449b-8f0a-cc4d406b10a7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:26.481722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007822f1-0480-419a-8336-d6ac1031638d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:26.494325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 900/1797] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:26 2022] PUT /identity/v3/projects/27eaea7979df4282b78601f3ac0cd441/users/df1b9e2b30694eb0a30e53f12906f016/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:26.496198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975711d0-a412-449b-8f0a-cc4d406b10a7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:26.666708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f363010-64d8-4bfe-b761-b80c23e6291d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:26.667900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f363010-64d8-4bfe-b761-b80c23e6291d service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:26.668478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f363010-64d8-4bfe-b761-b80c23e6291d service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:26.669579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f363010-64d8-4bfe-b761-b80c23e6291d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:26.697373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f363010-64d8-4bfe-b761-b80c23e6291d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:26.698984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f363010-64d8-4bfe-b761-b80c23e6291d service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:26.699638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975711d0-a412-449b-8f0a-cc4d406b10a7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:26.730411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 898/1798] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:26.731029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e3dcf-fd48-417d-9263-66e228f9a8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:26.733210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f363010-64d8-4bfe-b761-b80c23e6291d service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:26.734045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e3dcf-fd48-417d-9263-66e228f9a8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:26.734590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e3dcf-fd48-417d-9263-66e228f9a8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:26.734703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e3dcf-fd48-417d-9263-66e228f9a8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:26.737515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e3dcf-fd48-417d-9263-66e228f9a8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:26.747273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e3dcf-fd48-417d-9263-66e228f9a8cf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:26.756919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f363010-64d8-4bfe-b761-b80c23e6291d service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:26.764917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5969e-bdc5-447f-8bd9-f594505fdc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:26.765631 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5969e-bdc5-447f-8bd9-f594505fdc72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:26.766037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5969e-bdc5-447f-8bd9-f594505fdc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:26.766410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5969e-bdc5-447f-8bd9-f594505fdc72 admin admin] PATH_INFO: `/v3/projects/7765f0bee2b2436383c1805f87af3e33/users/9ac3492e69ae418b8d9a161f6472e340/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:26.770519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5969e-bdc5-447f-8bd9-f594505fdc72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7765f0bee2b2436383c1805f87af3e33, user_id=9ac3492e69ae418b8d9a161f6472e340, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:26.774317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5969e-bdc5-447f-8bd9-f594505fdc72 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:26.789561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 899/1799] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:26 2022] PUT /identity/v3/projects/7765f0bee2b2436383c1805f87af3e33/users/9ac3492e69ae418b8d9a161f6472e340/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:26.790800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fda45c-d55e-4c9f-977e-bfe1733f3f69 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:26.791147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fda45c-d55e-4c9f-977e-bfe1733f3f69 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:26.791558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fda45c-d55e-4c9f-977e-bfe1733f3f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:26.794358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 901/1800] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:26.798049 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e3dcf-fd48-417d-9263-66e228f9a8cf admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:26.844513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e3dcf-fd48-417d-9263-66e228f9a8cf admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:26.859558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:26.894805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-35fda45c-d55e-4c9f-977e-bfe1733f3f69 None None] MFA Rules not processed for user `df1b9e2b30694eb0a30e53f12906f016`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:26.906357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-35fda45c-d55e-4c9f-977e-bfe1733f3f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:26.910513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:26.975929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490f0c6-04a4-42c6-8081-c18b9f766051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:26.976591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490f0c6-04a4-42c6-8081-c18b9f766051 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:26.976827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490f0c6-04a4-42c6-8081-c18b9f766051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:26.983043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490f0c6-04a4-42c6-8081-c18b9f766051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:26.983903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490f0c6-04a4-42c6-8081-c18b9f766051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:26.985532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490f0c6-04a4-42c6-8081-c18b9f766051 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:27.002761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 902/1801] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:27.002761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05326cd-627f-42bd-98e8-11561fb097c5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:27.002761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05326cd-627f-42bd-98e8-11561fb097c5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:27.002761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05326cd-627f-42bd-98e8-11561fb097c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:27.052367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b05326cd-627f-42bd-98e8-11561fb097c5 None None] MFA Rules not processed for user `9ac3492e69ae418b8d9a161f6472e340`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:27.085390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b05326cd-627f-42bd-98e8-11561fb097c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:27.180738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 900/1802] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:27.183107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52afac-9f15-4eab-b762-d3203f640cf8 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:27.183436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52afac-9f15-4eab-b762-d3203f640cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:27.183717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52afac-9f15-4eab-b762-d3203f640cf8 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:27.187937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 901/1803] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 10:03:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:03:27.188708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52afac-9f15-4eab-b762-d3203f640cf8 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:27.226468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52afac-9f15-4eab-b762-d3203f640cf8 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:27.241990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bfb60-aee7-4d3d-a3d7-d02c4112911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:27.244664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bfb60-aee7-4d3d-a3d7-d02c4112911e admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:27.244954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bfb60-aee7-4d3d-a3d7-d02c4112911e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:27.245230 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bfb60-aee7-4d3d-a3d7-d02c4112911e admin admin] PATH_INFO: `/v3/projects/01f9eed7eb894574af165022ff8014ff/users/0bfc51f69ea74560bc3a9152f646428d/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:27.299460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bfb60-aee7-4d3d-a3d7-d02c4112911e admin admin] RBAC: Authorizing `identity:create_grant(project_id=01f9eed7eb894574af165022ff8014ff, user_id=0bfc51f69ea74560bc3a9152f646428d, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:27.303392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bfb60-aee7-4d3d-a3d7-d02c4112911e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:27.351591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 902/1804] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:27 2022] PUT /identity/v3/projects/01f9eed7eb894574af165022ff8014ff/users/0bfc51f69ea74560bc3a9152f646428d/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:27.351591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bfb60-aee7-4d3d-a3d7-d02c4112911e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:27.353294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-077bfb60-aee7-4d3d-a3d7-d02c4112911e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:27.457923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 903/1805] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:26 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:27.466142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e721710-fa29-402c-bb51-6bac0c055a26 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:27.469780 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e721710-fa29-402c-bb51-6bac0c055a26 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:27.470058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e721710-fa29-402c-bb51-6bac0c055a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:27.475274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bfb60-aee7-4d3d-a3d7-d02c4112911e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:27.486275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d10c24-665b-4036-9269-106da11b0a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:27.491416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d10c24-665b-4036-9269-106da11b0a92 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:27.491740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d10c24-665b-4036-9269-106da11b0a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:27.492012 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d10c24-665b-4036-9269-106da11b0a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:27.545433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9e721710-fa29-402c-bb51-6bac0c055a26 None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:27.549633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9e721710-fa29-402c-bb51-6bac0c055a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:27.697794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d10c24-665b-4036-9269-106da11b0a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:27.699884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d10c24-665b-4036-9269-106da11b0a92 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:27.737067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 903/1806] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:27.740374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a5291-d8d1-467f-a46b-3fe5403274ce None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:27.740854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a5291-d8d1-467f-a46b-3fe5403274ce None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:27.741312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a5291-d8d1-467f-a46b-3fe5403274ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:27.756769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 904/1807] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 10:03:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:27.833810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-dc8a5291-d8d1-467f-a46b-3fe5403274ce None None] MFA Rules not processed for user `0bfc51f69ea74560bc3a9152f646428d`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:27.834104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e721710-fa29-402c-bb51-6bac0c055a26 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:27.835976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-dc8a5291-d8d1-467f-a46b-3fe5403274ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:27.847382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e721710-fa29-402c-bb51-6bac0c055a26 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:27.863510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89877e8d-c2b8-4b94-8a51-b8e65fd4e3c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:27.864541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89877e8d-c2b8-4b94-8a51-b8e65fd4e3c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:27.864844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89877e8d-c2b8-4b94-8a51-b8e65fd4e3c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:27.865262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89877e8d-c2b8-4b94-8a51-b8e65fd4e3c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:27.881404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:28.049481 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:28.105282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 904/1808] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:28.109389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a5291-d8d1-467f-a46b-3fe5403274ce None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:28.116684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-dc8a5291-d8d1-467f-a46b-3fe5403274ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:28.122247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89877e8d-c2b8-4b94-8a51-b8e65fd4e3c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:28.125081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89877e8d-c2b8-4b94-8a51-b8e65fd4e3c8 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:28.157763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a5291-d8d1-467f-a46b-3fe5403274ce None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:28.166480 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 905/1809] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:27 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:28.171475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c3939-4681-4772-92cf-be00dbbfec69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:28.171987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89877e8d-c2b8-4b94-8a51-b8e65fd4e3c8 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:28.172493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c3939-4681-4772-92cf-be00dbbfec69 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:28.172956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c3939-4681-4772-92cf-be00dbbfec69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:28.173361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c3939-4681-4772-92cf-be00dbbfec69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:28.175673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-89877e8d-c2b8-4b94-8a51-b8e65fd4e3c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:28.177915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4c5c3939-4681-4772-92cf-be00dbbfec69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:28.272488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89877e8d-c2b8-4b94-8a51-b8e65fd4e3c8 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:28.319907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c9de9-b83b-411d-9fc5-738b6d357a82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:28.323425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c9de9-b83b-411d-9fc5-738b6d357a82 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:28.329196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c9de9-b83b-411d-9fc5-738b6d357a82 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:28.329769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c9de9-b83b-411d-9fc5-738b6d357a82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:28.329963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c3939-4681-4772-92cf-be00dbbfec69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:28.331615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c3939-4681-4772-92cf-be00dbbfec69 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:28.362112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c9de9-b83b-411d-9fc5-738b6d357a82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:28.365441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c9de9-b83b-411d-9fc5-738b6d357a82 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:28.375912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 905/1810] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:28.376868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c3939-4681-4772-92cf-be00dbbfec69 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:28.405490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 906/1811] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:28.628401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c3939-4681-4772-92cf-be00dbbfec69 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:28.646843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a7e8a-2510-4a2f-8a40-7b8fa88c0656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:28.647507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a7e8a-2510-4a2f-8a40-7b8fa88c0656 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:28.647787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a7e8a-2510-4a2f-8a40-7b8fa88c0656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:28.648061 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a7e8a-2510-4a2f-8a40-7b8fa88c0656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:28.648711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a7e8a-2510-4a2f-8a40-7b8fa88c0656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:28.650727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a7e8a-2510-4a2f-8a40-7b8fa88c0656 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:28.737129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 906/1812] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:28 2022] POST /identity/v3/projects => generated 377 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:28.741465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c9de9-b83b-411d-9fc5-738b6d357a82 service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:28.780296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c9de9-b83b-411d-9fc5-738b6d357a82 service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:28.783648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a7e8a-2510-4a2f-8a40-7b8fa88c0656 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:28.901300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:28.968239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381ac7e-d627-4e08-991d-5828440df131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:28.969078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381ac7e-d627-4e08-991d-5828440df131 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:28.969537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381ac7e-d627-4e08-991d-5828440df131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:28.969960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381ac7e-d627-4e08-991d-5828440df131 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:28.970861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381ac7e-d627-4e08-991d-5828440df131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:28.971701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a7e8a-2510-4a2f-8a40-7b8fa88c0656 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:28.974743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381ac7e-d627-4e08-991d-5828440df131 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:28.977510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73697b30-7c03-465c-a154-fe0592dbe901 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:28.978121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73697b30-7c03-465c-a154-fe0592dbe901 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:28.978403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73697b30-7c03-465c-a154-fe0592dbe901 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:28.978671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73697b30-7c03-465c-a154-fe0592dbe901 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:28.985379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-73697b30-7c03-465c-a154-fe0592dbe901 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:29.008458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 907/1813] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:28 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:29.015448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381ac7e-d627-4e08-991d-5828440df131 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:29.033060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381ac7e-d627-4e08-991d-5828440df131 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:29.040898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c56e7e-094e-481d-ae68-075905f8b979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:29.041779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c56e7e-094e-481d-ae68-075905f8b979 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:29.042206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c56e7e-094e-481d-ae68-075905f8b979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:29.042623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c56e7e-094e-481d-ae68-075905f8b979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:29.043484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c56e7e-094e-481d-ae68-075905f8b979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:29.046083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c56e7e-094e-481d-ae68-075905f8b979 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:29.060064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 908/1814] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:29.064289 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c56e7e-094e-481d-ae68-075905f8b979 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:29.087425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c56e7e-094e-481d-ae68-075905f8b979 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:29.103879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73697b30-7c03-465c-a154-fe0592dbe901 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:29.105454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73697b30-7c03-465c-a154-fe0592dbe901 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:29.106352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26bc17-19c1-472e-a24d-b6822cda3c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:29.106993 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26bc17-19c1-472e-a24d-b6822cda3c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:29.107808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26bc17-19c1-472e-a24d-b6822cda3c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:29.108078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26bc17-19c1-472e-a24d-b6822cda3c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:29.108812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26bc17-19c1-472e-a24d-b6822cda3c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:29.110469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26bc17-19c1-472e-a24d-b6822cda3c0d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:29.119263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 907/1815] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:28 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:29.135848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 909/1816] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:29.140515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73697b30-7c03-465c-a154-fe0592dbe901 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:29.158159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73697b30-7c03-465c-a154-fe0592dbe901 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:29.172403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b718115-cff0-49f9-80e9-b7187207cab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:29.175387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b718115-cff0-49f9-80e9-b7187207cab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:29.175910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b718115-cff0-49f9-80e9-b7187207cab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:29.176035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b718115-cff0-49f9-80e9-b7187207cab7 admin admin] PATH_INFO: `/v3/projects/690f7f247fef43969a37c9fde0253d18/users/bc63490de1414f9aa8f28ad2248f58ee/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:29.211526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b718115-cff0-49f9-80e9-b7187207cab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690f7f247fef43969a37c9fde0253d18, user_id=bc63490de1414f9aa8f28ad2248f58ee, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:29.216023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b718115-cff0-49f9-80e9-b7187207cab7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:29.235110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 908/1817] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:29 2022] PUT /identity/v3/projects/690f7f247fef43969a37c9fde0253d18/users/bc63490de1414f9aa8f28ad2248f58ee/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:29.244973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8a57c-f0c3-48a7-b71c-1534a53d033f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:29.245351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8a57c-f0c3-48a7-b71c-1534a53d033f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:29.245576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8a57c-f0c3-48a7-b71c-1534a53d033f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:29.257499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b718115-cff0-49f9-80e9-b7187207cab7 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:29.259825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9b718115-cff0-49f9-80e9-b7187207cab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:29.311353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-96a8a57c-f0c3-48a7-b71c-1534a53d033f None None] MFA Rules not processed for user `bc63490de1414f9aa8f28ad2248f58ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:29.312584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-96a8a57c-f0c3-48a7-b71c-1534a53d033f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:29.423077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b718115-cff0-49f9-80e9-b7187207cab7 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:29.432854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:29.433961 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:29.434816 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:29.436026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] PATH_INFO: `/v3/projects/90bcc8b088e34fb98186f4b1faa47983` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:29.438032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90bcc8b088e34fb98186f4b1faa47983)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:29.440394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:29.474751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 910/1818] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:29.480873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8a57c-f0c3-48a7-b71c-1534a53d033f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:29.500898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8a57c-f0c3-48a7-b71c-1534a53d033f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:29.509711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90bcc8b088e34fb98186f4b1faa47983', 'request_id': 'req-7077109b-cc20-42da-95bd-66541177d346'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.519190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90bcc8b088e34fb98186f4b1faa47983', 'request_id': 'req-7077109b-cc20-42da-95bd-66541177d346'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.532004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90bcc8b088e34fb98186f4b1faa47983', 'request_id': 'req-7077109b-cc20-42da-95bd-66541177d346'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.532212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:29.544913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:29.558549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] The token cache is being invalidate because project 90bcc8b088e34fb98186f4b1faa47983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:29.559334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.561169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.567036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 909/1819] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:29 2022] DELETE /identity/v3/projects/90bcc8b088e34fb98186f4b1faa47983 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:29.570848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:29.572847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:29.607032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d65222-7ed3-4f12-9da9-daf247449090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:29.608009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d65222-7ed3-4f12-9da9-daf247449090 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:29.609405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d65222-7ed3-4f12-9da9-daf247449090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:29.609790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d65222-7ed3-4f12-9da9-daf247449090 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:29.610704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d65222-7ed3-4f12-9da9-daf247449090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:29.614678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d65222-7ed3-4f12-9da9-daf247449090 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:29.657858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 911/1820] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:29 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:29.661242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077109b-cc20-42da-95bd-66541177d346 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:29.672788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d65222-7ed3-4f12-9da9-daf247449090 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:29.677109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b7d65222-7ed3-4f12-9da9-daf247449090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:29.755690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:29.756528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:29.756807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:29.757392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] PATH_INFO: `/v3/users/ffb17d1b3f444be78a8acc8bd151d101` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:29.758315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb17d1b3f444be78a8acc8bd151d101)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:29.760364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:29.769541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d65222-7ed3-4f12-9da9-daf247449090 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:29.774592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e82e0-4188-4498-a379-51b084e2ee8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:29.775343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e82e0-4188-4498-a379-51b084e2ee8e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:29.775561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e82e0-4188-4498-a379-51b084e2ee8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:29.775833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e82e0-4188-4498-a379-51b084e2ee8e admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:29.776454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e82e0-4188-4498-a379-51b084e2ee8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:29.778691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e82e0-4188-4498-a379-51b084e2ee8e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:29.817851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 912/1821] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:29 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:29.822181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e82e0-4188-4498-a379-51b084e2ee8e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:29.826696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb17d1b3f444be78a8acc8bd151d101'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.845145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e82e0-4188-4498-a379-51b084e2ee8e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:29.845787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb17d1b3f444be78a8acc8bd151d101'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.854349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb17d1b3f444be78a8acc8bd151d101'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.855904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3868f8-db92-488c-8a75-7aeca4d0cb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:29.858083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3868f8-db92-488c-8a75-7aeca4d0cb97 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:29.858579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3868f8-db92-488c-8a75-7aeca4d0cb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:29.859130 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3868f8-db92-488c-8a75-7aeca4d0cb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:29.860095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3868f8-db92-488c-8a75-7aeca4d0cb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:29.864085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3868f8-db92-488c-8a75-7aeca4d0cb97 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:29.867860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb17d1b3f444be78a8acc8bd151d101'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.868588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb17d1b3f444be78a8acc8bd151d101'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:29.875650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 910/1822] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:29 2022] DELETE /identity/v3/users/ffb17d1b3f444be78a8acc8bd151d101 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:29.883219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 913/1823] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:29.891769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:29.895336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:29.973222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:30.052527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54025d45-31c8-4d1a-a6fa-bd57af48d862 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:30.058741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb56b5-64c1-4efa-bd99-bff01be0b8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:30.059857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb56b5-64c1-4efa-bd99-bff01be0b8c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.060957 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb56b5-64c1-4efa-bd99-bff01be0b8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.061285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb56b5-64c1-4efa-bd99-bff01be0b8c0 admin admin] PATH_INFO: `/v3/projects/4360d19275514f4286485fea2af27414/users/d1c2528bd5c448d3ad721f55525dbac1/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.083103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb56b5-64c1-4efa-bd99-bff01be0b8c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4360d19275514f4286485fea2af27414, user_id=d1c2528bd5c448d3ad721f55525dbac1, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:30.085190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb56b5-64c1-4efa-bd99-bff01be0b8c0 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:30.109792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 911/1824] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:29 2022] PUT /identity/v3/projects/4360d19275514f4286485fea2af27414/users/d1c2528bd5c448d3ad721f55525dbac1/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:30.111500 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3868f8-db92-488c-8a75-7aeca4d0cb97 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:30.127885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3868f8-db92-488c-8a75-7aeca4d0cb97 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:30.209081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c49fd-aa84-4c2c-8c35-d958b172ef7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:30.209826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c49fd-aa84-4c2c-8c35-d958b172ef7e admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.210160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c49fd-aa84-4c2c-8c35-d958b172ef7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.210640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c49fd-aa84-4c2c-8c35-d958b172ef7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.211465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c49fd-aa84-4c2c-8c35-d958b172ef7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:30.213194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c49fd-aa84-4c2c-8c35-d958b172ef7e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:30.241174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 914/1825] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:30.242487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb56b5-64c1-4efa-bd99-bff01be0b8c0 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:30.262868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb56b5-64c1-4efa-bd99-bff01be0b8c0 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:30.279467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db806d19-255c-4c71-8852-6fa222ae4096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:30.279467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db806d19-255c-4c71-8852-6fa222ae4096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.279467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db806d19-255c-4c71-8852-6fa222ae4096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.279467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db806d19-255c-4c71-8852-6fa222ae4096 admin admin] PATH_INFO: `/v3/projects/4360d19275514f4286485fea2af27414/users/d1c2528bd5c448d3ad721f55525dbac1/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.283677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db806d19-255c-4c71-8852-6fa222ae4096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4360d19275514f4286485fea2af27414, user_id=d1c2528bd5c448d3ad721f55525dbac1, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:30.290354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db806d19-255c-4c71-8852-6fa222ae4096 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:30.302976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 912/1826] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:30 2022] PUT /identity/v3/projects/4360d19275514f4286485fea2af27414/users/d1c2528bd5c448d3ad721f55525dbac1/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:30.313226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea8966-bb6f-498e-a61d-bc2ec8597148 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.313515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea8966-bb6f-498e-a61d-bc2ec8597148 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.313812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea8966-bb6f-498e-a61d-bc2ec8597148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.369614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f9ea8966-bb6f-498e-a61d-bc2ec8597148 None None] MFA Rules not processed for user `d1c2528bd5c448d3ad721f55525dbac1`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:30.371712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f9ea8966-bb6f-498e-a61d-bc2ec8597148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:30.389098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:30.472096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 915/1827] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:30.484585 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db806d19-255c-4c71-8852-6fa222ae4096 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:30.486297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-db806d19-255c-4c71-8852-6fa222ae4096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:30.552523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db806d19-255c-4c71-8852-6fa222ae4096 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:30.556803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1c738-40ea-4539-a6a7-919f0562eb98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:30.557678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1c738-40ea-4539-a6a7-919f0562eb98 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.558071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1c738-40ea-4539-a6a7-919f0562eb98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.558484 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1c738-40ea-4539-a6a7-919f0562eb98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.566067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea8966-bb6f-498e-a61d-bc2ec8597148 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:30.567235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f9ea8966-bb6f-498e-a61d-bc2ec8597148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:30.575002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1c738-40ea-4539-a6a7-919f0562eb98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:30.577022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1c738-40ea-4539-a6a7-919f0562eb98 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:30.602872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 913/1828] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:30.641075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1c738-40ea-4539-a6a7-919f0562eb98 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:30.643263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-34f1c738-40ea-4539-a6a7-919f0562eb98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:30.657493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1c738-40ea-4539-a6a7-919f0562eb98 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:30.665705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb6952-ec08-4b83-91e2-85c6a6d7d95a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:30.666514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6952-ec08-4b83-91e2-85c6a6d7d95a service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.666869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6952-ec08-4b83-91e2-85c6a6d7d95a service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.667302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6952-ec08-4b83-91e2-85c6a6d7d95a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.670143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4dfb6952-ec08-4b83-91e2-85c6a6d7d95a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:30.676758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea8966-bb6f-498e-a61d-bc2ec8597148 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:30.681464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:30.682097 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.682377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.682649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] PATH_INFO: `/v3/projects/180353c3abc04d35b6c6b3e78be90979` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.683789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=180353c3abc04d35b6c6b3e78be90979)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:30.685785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:30.734716 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180353c3abc04d35b6c6b3e78be90979', 'request_id': 'req-1168b779-954a-4166-abb4-741ba43a0fa4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:30.742496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180353c3abc04d35b6c6b3e78be90979', 'request_id': 'req-1168b779-954a-4166-abb4-741ba43a0fa4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:30.744639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb6952-ec08-4b83-91e2-85c6a6d7d95a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:30.746107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb6952-ec08-4b83-91e2-85c6a6d7d95a service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:30.749700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180353c3abc04d35b6c6b3e78be90979', 'request_id': 'req-1168b779-954a-4166-abb4-741ba43a0fa4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:30.756114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 914/1829] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:30.760676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] The token cache is being invalidate because project 180353c3abc04d35b6c6b3e78be90979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:30.761024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:30.761546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1168b779-954a-4166-abb4-741ba43a0fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:30.763083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 916/1830] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:30 2022] DELETE /identity/v3/projects/180353c3abc04d35b6c6b3e78be90979 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:30.768840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bb806-a1c8-4724-960f-bfa7f0e64c22 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.769232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bb806-a1c8-4724-960f-bfa7f0e64c22 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.769611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bb806-a1c8-4724-960f-bfa7f0e64c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.789653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-330bb806-a1c8-4724-960f-bfa7f0e64c22 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:30.791045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-330bb806-a1c8-4724-960f-bfa7f0e64c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:30.822076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:30.860671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 915/1831] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:30.862363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ca71-b866-425f-afd6-0adcdb13826f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.862655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ca71-b866-425f-afd6-0adcdb13826f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.862934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ca71-b866-425f-afd6-0adcdb13826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.882612 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e127ca71-b866-425f-afd6-0adcdb13826f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:30.883861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e127ca71-b866-425f-afd6-0adcdb13826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:30.892545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 917/1832] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:30.897208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bb806-a1c8-4724-960f-bfa7f0e64c22 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:30.913363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bb806-a1c8-4724-960f-bfa7f0e64c22 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:30.919538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b411336-bec9-4eb8-9c17-a86f1fb05ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:30.920168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b411336-bec9-4eb8-9c17-a86f1fb05ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.920448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b411336-bec9-4eb8-9c17-a86f1fb05ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.922926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b411336-bec9-4eb8-9c17-a86f1fb05ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.924032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b411336-bec9-4eb8-9c17-a86f1fb05ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:30.925142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b411336-bec9-4eb8-9c17-a86f1fb05ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:30.926917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b411336-bec9-4eb8-9c17-a86f1fb05ed3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:30.949728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 916/1833] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:30.950994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127ca71-b866-425f-afd6-0adcdb13826f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:30.961437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127ca71-b866-425f-afd6-0adcdb13826f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:30.967031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac40449-582d-4150-93bc-9c43a607b4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:30.967648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac40449-582d-4150-93bc-9c43a607b4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:30.967927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac40449-582d-4150-93bc-9c43a607b4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:30.968493 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac40449-582d-4150-93bc-9c43a607b4d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:30.969176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac40449-582d-4150-93bc-9c43a607b4d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:30.971472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac40449-582d-4150-93bc-9c43a607b4d2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:31.020577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 918/1834] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:30 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:31.026112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b411336-bec9-4eb8-9c17-a86f1fb05ed3 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:31.044206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b411336-bec9-4eb8-9c17-a86f1fb05ed3 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:31.114593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfb4db-a27f-47d6-aa8d-adb2c31a5837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:31.115641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfb4db-a27f-47d6-aa8d-adb2c31a5837 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:31.116134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfb4db-a27f-47d6-aa8d-adb2c31a5837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:31.116572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfb4db-a27f-47d6-aa8d-adb2c31a5837 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:31.117506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfb4db-a27f-47d6-aa8d-adb2c31a5837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:31.120000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfb4db-a27f-47d6-aa8d-adb2c31a5837 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:31.150860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 917/1835] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:31 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:31.155656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac40449-582d-4150-93bc-9c43a607b4d2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:31.176693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac40449-582d-4150-93bc-9c43a607b4d2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:31.186052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca90b9-3e05-4364-80c7-396a5112909c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:31.186832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca90b9-3e05-4364-80c7-396a5112909c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:31.187168 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca90b9-3e05-4364-80c7-396a5112909c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:31.187523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca90b9-3e05-4364-80c7-396a5112909c admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:31.189825 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca90b9-3e05-4364-80c7-396a5112909c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:31.190006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca90b9-3e05-4364-80c7-396a5112909c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:31.206501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:31.222240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 919/1836] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:31.224072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfb4db-a27f-47d6-aa8d-adb2c31a5837 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:31.241064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfb4db-a27f-47d6-aa8d-adb2c31a5837 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:31.284054 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbccb8-e962-4f49-a216-af14c499f50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:31.285361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbccb8-e962-4f49-a216-af14c499f50b admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:31.286106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbccb8-e962-4f49-a216-af14c499f50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:31.286704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbccb8-e962-4f49-a216-af14c499f50b admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:31.287890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbccb8-e962-4f49-a216-af14c499f50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:31.290344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbccb8-e962-4f49-a216-af14c499f50b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:31.303358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 918/1837] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:31.307646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca90b9-3e05-4364-80c7-396a5112909c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:31.324539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca90b9-3e05-4364-80c7-396a5112909c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:31.331245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe85d91-3d25-4758-89b0-65effa70442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:31.332141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe85d91-3d25-4758-89b0-65effa70442d admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:31.332438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe85d91-3d25-4758-89b0-65effa70442d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:31.332878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe85d91-3d25-4758-89b0-65effa70442d admin admin] PATH_INFO: `/v3/projects/d54ef161fae444098be5bac0f1156118/users/bb88d9d16c8c4c37a19fd439a516ba1a/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:31.366024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe85d91-3d25-4758-89b0-65effa70442d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54ef161fae444098be5bac0f1156118, user_id=bb88d9d16c8c4c37a19fd439a516ba1a, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:31.371107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe85d91-3d25-4758-89b0-65effa70442d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:31.396635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 920/1838] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:31 2022] PUT /identity/v3/projects/d54ef161fae444098be5bac0f1156118/users/bb88d9d16c8c4c37a19fd439a516ba1a/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:31.408758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcc9f3-d68e-46c2-9eff-7d11eff768ee None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:31.409041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcc9f3-d68e-46c2-9eff-7d11eff768ee None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:31.409402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcc9f3-d68e-46c2-9eff-7d11eff768ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:31.468657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-eadcc9f3-d68e-46c2-9eff-7d11eff768ee None None] MFA Rules not processed for user `bb88d9d16c8c4c37a19fd439a516ba1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:31.471061 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-eadcc9f3-d68e-46c2-9eff-7d11eff768ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:31.662050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 919/1839] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:31.666533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe85d91-3d25-4758-89b0-65effa70442d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:31.850405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe85d91-3d25-4758-89b0-65effa70442d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:31.859610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859bfd-8b5e-41ec-b29c-3f12accd8170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:31.860226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859bfd-8b5e-41ec-b29c-3f12accd8170 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:31.861353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859bfd-8b5e-41ec-b29c-3f12accd8170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:31.861353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859bfd-8b5e-41ec-b29c-3f12accd8170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:31.861564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb859bfd-8b5e-41ec-b29c-3f12accd8170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:31.863469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb859bfd-8b5e-41ec-b29c-3f12accd8170 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:31.979427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 921/1840] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:31 2022] POST /identity/v3/projects => generated 387 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:31.983604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcc9f3-d68e-46c2-9eff-7d11eff768ee None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:32.012387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcc9f3-d68e-46c2-9eff-7d11eff768ee None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:32.084599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:32.153276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3fdf4-71ae-43e2-a297-86b6e57f771f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:32.153957 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3fdf4-71ae-43e2-a297-86b6e57f771f admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:32.154736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3fdf4-71ae-43e2-a297-86b6e57f771f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:32.155008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3fdf4-71ae-43e2-a297-86b6e57f771f admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:32.155647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3fdf4-71ae-43e2-a297-86b6e57f771f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:32.157627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3fdf4-71ae-43e2-a297-86b6e57f771f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:32.209131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 920/1841] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:31 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:32.209948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859bfd-8b5e-41ec-b29c-3f12accd8170 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:32.236629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859bfd-8b5e-41ec-b29c-3f12accd8170 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:32.257922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc856be-4121-4772-b1cd-18b1496c4713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:32.258545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc856be-4121-4772-b1cd-18b1496c4713 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:32.259038 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc856be-4121-4772-b1cd-18b1496c4713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:32.259362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc856be-4121-4772-b1cd-18b1496c4713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:32.260232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc856be-4121-4772-b1cd-18b1496c4713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:32.271212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc856be-4121-4772-b1cd-18b1496c4713 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:32.319595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 922/1842] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:32.323022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3fdf4-71ae-43e2-a297-86b6e57f771f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:32.346924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3fdf4-71ae-43e2-a297-86b6e57f771f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:32.368257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6f5d9-5672-4f25-a7fa-aafee7f8bb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:32.368580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6f5d9-5672-4f25-a7fa-aafee7f8bb17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:32.368876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6f5d9-5672-4f25-a7fa-aafee7f8bb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:32.369130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6f5d9-5672-4f25-a7fa-aafee7f8bb17 admin admin] PATH_INFO: `/v3/projects/63271b516ea846d1881a30eeb39b24db/users/cb003e77c0d0497faf2b979b883578fb/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:32.448818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6f5d9-5672-4f25-a7fa-aafee7f8bb17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63271b516ea846d1881a30eeb39b24db, user_id=cb003e77c0d0497faf2b979b883578fb, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:32.455743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6f5d9-5672-4f25-a7fa-aafee7f8bb17 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:32.525979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 921/1843] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:32 2022] PUT /identity/v3/projects/63271b516ea846d1881a30eeb39b24db/users/cb003e77c0d0497faf2b979b883578fb/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:32.533778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc856be-4121-4772-b1cd-18b1496c4713 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:32.554800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc856be-4121-4772-b1cd-18b1496c4713 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:32.786684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc20de0-58af-498a-901c-293b8f435983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:32.787345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc20de0-58af-498a-901c-293b8f435983 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:32.787864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc20de0-58af-498a-901c-293b8f435983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:32.788297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc20de0-58af-498a-901c-293b8f435983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:32.789101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc20de0-58af-498a-901c-293b8f435983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:32.790865 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc20de0-58af-498a-901c-293b8f435983 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:32.804097 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 923/1844] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:32.808008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6f5d9-5672-4f25-a7fa-aafee7f8bb17 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:32.825542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6f5d9-5672-4f25-a7fa-aafee7f8bb17 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:32.833849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e6154-c6b0-46d1-bdb5-6db9ce1afc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:32.835163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e6154-c6b0-46d1-bdb5-6db9ce1afc0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:32.835803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e6154-c6b0-46d1-bdb5-6db9ce1afc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:32.836235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e6154-c6b0-46d1-bdb5-6db9ce1afc0f admin admin] PATH_INFO: `/v3/projects/63271b516ea846d1881a30eeb39b24db/users/cb003e77c0d0497faf2b979b883578fb/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:32.838630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e6154-c6b0-46d1-bdb5-6db9ce1afc0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=63271b516ea846d1881a30eeb39b24db, user_id=cb003e77c0d0497faf2b979b883578fb, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:32.843138 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e6154-c6b0-46d1-bdb5-6db9ce1afc0f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:32.858103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 922/1845] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:32 2022] PUT /identity/v3/projects/63271b516ea846d1881a30eeb39b24db/users/cb003e77c0d0497faf2b979b883578fb/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:32.868169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848c2b6-8ae9-4bbd-b8c6-f3102bb8713a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:32.868815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848c2b6-8ae9-4bbd-b8c6-f3102bb8713a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:32.869250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848c2b6-8ae9-4bbd-b8c6-f3102bb8713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:32.929733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2848c2b6-8ae9-4bbd-b8c6-f3102bb8713a None None] MFA Rules not processed for user `cb003e77c0d0497faf2b979b883578fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:32.933254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2848c2b6-8ae9-4bbd-b8c6-f3102bb8713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:32.999273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:33.076577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 924/1846] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:33.126372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e6154-c6b0-46d1-bdb5-6db9ce1afc0f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:33.128287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f35e6154-c6b0-46d1-bdb5-6db9ce1afc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:33.197995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e6154-c6b0-46d1-bdb5-6db9ce1afc0f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:33.207835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a9fb6-efac-45ce-a34c-eb4c8b286966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:33.208920 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a9fb6-efac-45ce-a34c-eb4c8b286966 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:33.209469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a9fb6-efac-45ce-a34c-eb4c8b286966 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:33.209938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a9fb6-efac-45ce-a34c-eb4c8b286966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:33.220629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a9fb6-efac-45ce-a34c-eb4c8b286966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:33.222390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a9fb6-efac-45ce-a34c-eb4c8b286966 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:33.234613 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 923/1847] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:33.276538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848c2b6-8ae9-4bbd-b8c6-f3102bb8713a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:33.416667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848c2b6-8ae9-4bbd-b8c6-f3102bb8713a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:33.427442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:33.428262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:33.428547 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:33.428818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] PATH_INFO: `/v3/users/bb88d9d16c8c4c37a19fd439a516ba1a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:33.430005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb88d9d16c8c4c37a19fd439a516ba1a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:33.432070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:33.540655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb88d9d16c8c4c37a19fd439a516ba1a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:33.555404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb88d9d16c8c4c37a19fd439a516ba1a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:33.583847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb88d9d16c8c4c37a19fd439a516ba1a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:33.589807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb88d9d16c8c4c37a19fd439a516ba1a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:33.590572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2db6bda4-4265-4ef3-9b87-fb1ba5300b8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb88d9d16c8c4c37a19fd439a516ba1a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:33.618306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 925/1848] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:33 2022] DELETE /identity/v3/users/bb88d9d16c8c4c37a19fd439a516ba1a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:33.620128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a7bb1-6d68-44d7-859d-001bb9274b75 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:33.620712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a7bb1-6d68-44d7-859d-001bb9274b75 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:33.621140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a7bb1-6d68-44d7-859d-001bb9274b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:33.683046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-080a7bb1-6d68-44d7-859d-001bb9274b75 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:33.684427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-080a7bb1-6d68-44d7-859d-001bb9274b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:33.743089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c694c0b-36f9-435f-ad0a-b918c9581539 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:33.744321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c694c0b-36f9-435f-ad0a-b918c9581539 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:33.744991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c694c0b-36f9-435f-ad0a-b918c9581539 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:33.746853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 926/1849] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 10:03:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:03:33.812130 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93af51-9eb0-4b49-9a9e-25ce61e271bf None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:33.812628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93af51-9eb0-4b49-9a9e-25ce61e271bf None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:33.813039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93af51-9eb0-4b49-9a9e-25ce61e271bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:33.824057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 924/1850] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:33.840014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a7bb1-6d68-44d7-859d-001bb9274b75 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:33.841681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-080a7bb1-6d68-44d7-859d-001bb9274b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:33.850610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ad93af51-9eb0-4b49-9a9e-25ce61e271bf None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:33.855016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ad93af51-9eb0-4b49-9a9e-25ce61e271bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:33.880234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:33.936058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:33.949976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a7bb1-6d68-44d7-859d-001bb9274b75 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:33.955617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de802ac-3576-443c-88b9-0a1e25a824cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:33.956799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de802ac-3576-443c-88b9-0a1e25a824cd service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:33.957592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de802ac-3576-443c-88b9-0a1e25a824cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:33.958373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de802ac-3576-443c-88b9-0a1e25a824cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:33.965483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 927/1851] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 10:03:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:33.970580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de802ac-3576-443c-88b9-0a1e25a824cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:33.972330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de802ac-3576-443c-88b9-0a1e25a824cd service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:33.978782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93af51-9eb0-4b49-9a9e-25ce61e271bf None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:33.979928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ad93af51-9eb0-4b49-9a9e-25ce61e271bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:33.984323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 925/1852] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:34.002321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93af51-9eb0-4b49-9a9e-25ce61e271bf None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:34.009956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308af80-04e9-4fd5-baf9-c8ba409445e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:34.011293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308af80-04e9-4fd5-baf9-c8ba409445e6 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:34.011776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308af80-04e9-4fd5-baf9-c8ba409445e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:34.018947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308af80-04e9-4fd5-baf9-c8ba409445e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:34.024748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308af80-04e9-4fd5-baf9-c8ba409445e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:34.026366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308af80-04e9-4fd5-baf9-c8ba409445e6 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:34.040563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 928/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:34.352966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de802ac-3576-443c-88b9-0a1e25a824cd service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:34.355009 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-2de802ac-3576-443c-88b9-0a1e25a824cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:34.368717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de802ac-3576-443c-88b9-0a1e25a824cd service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:34.374285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:34.375135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:34.375787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:34.376069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] PATH_INFO: `/v3/projects/d54ef161fae444098be5bac0f1156118` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:34.377208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54ef161fae444098be5bac0f1156118)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:34.379304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:34.434909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54ef161fae444098be5bac0f1156118', 'request_id': 'req-c0c56969-e2db-4f00-8685-3655736c3483'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.444083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54ef161fae444098be5bac0f1156118', 'request_id': 'req-c0c56969-e2db-4f00-8685-3655736c3483'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.452031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54ef161fae444098be5bac0f1156118', 'request_id': 'req-c0c56969-e2db-4f00-8685-3655736c3483'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.462572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] The token cache is being invalidate because project d54ef161fae444098be5bac0f1156118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:34.462862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.463541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.465172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 926/1854] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:34 2022] DELETE /identity/v3/projects/d54ef161fae444098be5bac0f1156118 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:34.469645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308af80-04e9-4fd5-baf9-c8ba409445e6 service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:34.480549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308af80-04e9-4fd5-baf9-c8ba409445e6 service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:34.542102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:34.542770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:34.543093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:34.543366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] PATH_INFO: `/v3/users/cb003e77c0d0497faf2b979b883578fb` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:34.545055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb003e77c0d0497faf2b979b883578fb)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:34.546973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:34.596592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb003e77c0d0497faf2b979b883578fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.604589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb003e77c0d0497faf2b979b883578fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.617057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb003e77c0d0497faf2b979b883578fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.617785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb003e77c0d0497faf2b979b883578fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.618412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00ffae59-9c5e-47f5-aa5b-0f4331d3cdb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb003e77c0d0497faf2b979b883578fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:34.632472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 929/1855] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:34 2022] DELETE /identity/v3/users/cb003e77c0d0497faf2b979b883578fb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:35.015255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:35.016870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:35.031368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c56969-e2db-4f00-8685-3655736c3483 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:35.087332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:35.087963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.088242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.088515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] PATH_INFO: `/v3/projects/63271b516ea846d1881a30eeb39b24db` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.089615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] RBAC: Authorizing `identity:delete_project(project_id=63271b516ea846d1881a30eeb39b24db)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:35.091602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:35.136808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63271b516ea846d1881a30eeb39b24db', 'request_id': 'req-653b134f-eadc-4d22-8885-008b18b9297a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:35.138592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:35.142976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63271b516ea846d1881a30eeb39b24db', 'request_id': 'req-653b134f-eadc-4d22-8885-008b18b9297a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:35.150904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63271b516ea846d1881a30eeb39b24db', 'request_id': 'req-653b134f-eadc-4d22-8885-008b18b9297a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:35.162350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] The token cache is being invalidate because project 63271b516ea846d1881a30eeb39b24db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:35.162690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:35.163352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-653b134f-eadc-4d22-8885-008b18b9297a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:35.165017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 927/1856] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:35 2022] DELETE /identity/v3/projects/63271b516ea846d1881a30eeb39b24db => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:35.170581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4283ad-bca9-49c2-aa5d-3be062ac2500 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.170889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4283ad-bca9-49c2-aa5d-3be062ac2500 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.171218 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4283ad-bca9-49c2-aa5d-3be062ac2500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.194426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-db4283ad-bca9-49c2-aa5d-3be062ac2500 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:35.195811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-db4283ad-bca9-49c2-aa5d-3be062ac2500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:35.251421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:35.271471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 930/1857] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:35.279472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1ee97-0f61-4d95-bcea-0b8b35e8aedb None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.279756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1ee97-0f61-4d95-bcea-0b8b35e8aedb None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.280025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1ee97-0f61-4d95-bcea-0b8b35e8aedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.301470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-19d1ee97-0f61-4d95-bcea-0b8b35e8aedb None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:35.302696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-19d1ee97-0f61-4d95-bcea-0b8b35e8aedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:35.315058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 928/1858] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:35.319411 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4283ad-bca9-49c2-aa5d-3be062ac2500 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:35.329681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4283ad-bca9-49c2-aa5d-3be062ac2500 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:35.334335 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2d7d7-93dd-4511-a3f7-48b3d0540583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:35.334988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2d7d7-93dd-4511-a3f7-48b3d0540583 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.335279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2d7d7-93dd-4511-a3f7-48b3d0540583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.335568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2d7d7-93dd-4511-a3f7-48b3d0540583 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.336192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2d7d7-93dd-4511-a3f7-48b3d0540583 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:35.336492 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2d7d7-93dd-4511-a3f7-48b3d0540583 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:35.338088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2d7d7-93dd-4511-a3f7-48b3d0540583 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:35.352583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 931/1859] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:03:35.357865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1ee97-0f61-4d95-bcea-0b8b35e8aedb None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:35.367563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1ee97-0f61-4d95-bcea-0b8b35e8aedb None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:35.373386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a1ae2-9645-4ef3-86a7-09eb92e5cb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:35.373983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a1ae2-9645-4ef3-86a7-09eb92e5cb35 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.374489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a1ae2-9645-4ef3-86a7-09eb92e5cb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.374763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a1ae2-9645-4ef3-86a7-09eb92e5cb35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.375731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a1ae2-9645-4ef3-86a7-09eb92e5cb35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:35.377748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a1ae2-9645-4ef3-86a7-09eb92e5cb35 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:35.433419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 929/1860] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:35 2022] POST /identity/v3/projects => generated 393 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:35.438201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2d7d7-93dd-4511-a3f7-48b3d0540583 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:35.449095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2d7d7-93dd-4511-a3f7-48b3d0540583 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:35.513668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c996e7d-5fd0-455e-b102-5ce58c927fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:35.515366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c996e7d-5fd0-455e-b102-5ce58c927fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.515664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c996e7d-5fd0-455e-b102-5ce58c927fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.515941 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c996e7d-5fd0-455e-b102-5ce58c927fc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.516591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c996e7d-5fd0-455e-b102-5ce58c927fc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:35.518614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c996e7d-5fd0-455e-b102-5ce58c927fc3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:35.551740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 932/1861] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:35 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:35.553132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a1ae2-9645-4ef3-86a7-09eb92e5cb35 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:35.563678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a1ae2-9645-4ef3-86a7-09eb92e5cb35 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:35.568896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79e5c9-c547-4f79-ad29-5e408f0dbd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:35.569495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79e5c9-c547-4f79-ad29-5e408f0dbd6a admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.569773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79e5c9-c547-4f79-ad29-5e408f0dbd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.570045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79e5c9-c547-4f79-ad29-5e408f0dbd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.570687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79e5c9-c547-4f79-ad29-5e408f0dbd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:35.572310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79e5c9-c547-4f79-ad29-5e408f0dbd6a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:35.582989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 930/1862] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:35.586421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c996e7d-5fd0-455e-b102-5ce58c927fc3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:35.596973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c996e7d-5fd0-455e-b102-5ce58c927fc3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:35.601498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2718b-2ec2-47bd-9957-01e515f3f9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:35.602139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2718b-2ec2-47bd-9957-01e515f3f9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.602400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2718b-2ec2-47bd-9957-01e515f3f9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.602673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2718b-2ec2-47bd-9957-01e515f3f9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.603418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2718b-2ec2-47bd-9957-01e515f3f9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:35.604995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2718b-2ec2-47bd-9957-01e515f3f9ca admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:35.616333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 933/1863] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:35.620618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79e5c9-c547-4f79-ad29-5e408f0dbd6a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:35.630509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79e5c9-c547-4f79-ad29-5e408f0dbd6a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:35.635169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f3316-ca48-445b-844d-c022e52af80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:35.635904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3316-ca48-445b-844d-c022e52af80c admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.636185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3316-ca48-445b-844d-c022e52af80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.636460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3316-ca48-445b-844d-c022e52af80c admin admin] PATH_INFO: `/v3/projects/691f28dd9af54c6db82585fffef29c56/users/ff04af971ce94e2782c1f6d22a548ce4/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.650431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f3316-ca48-445b-844d-c022e52af80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=691f28dd9af54c6db82585fffef29c56, user_id=ff04af971ce94e2782c1f6d22a548ce4, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:35.653045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f3316-ca48-445b-844d-c022e52af80c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:35.665896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 931/1864] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:35 2022] PUT /identity/v3/projects/691f28dd9af54c6db82585fffef29c56/users/ff04af971ce94e2782c1f6d22a548ce4/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:35.676855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721323b7-552e-43c1-beb3-e61a02563a76 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.677146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721323b7-552e-43c1-beb3-e61a02563a76 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.677429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721323b7-552e-43c1-beb3-e61a02563a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.703812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-721323b7-552e-43c1-beb3-e61a02563a76 None None] MFA Rules not processed for user `ff04af971ce94e2782c1f6d22a548ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:35.705037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-721323b7-552e-43c1-beb3-e61a02563a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:35.769031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 934/1865] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:35.773504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f3316-ca48-445b-844d-c022e52af80c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:35.834161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f3316-ca48-445b-844d-c022e52af80c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:35.838833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3c78-a299-4690-b4f3-7c007e5e29c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:35.839473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3c78-a299-4690-b4f3-7c007e5e29c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.839754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3c78-a299-4690-b4f3-7c007e5e29c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.840032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3c78-a299-4690-b4f3-7c007e5e29c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.840657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef3c78-a299-4690-b4f3-7c007e5e29c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:35.843149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef3c78-a299-4690-b4f3-7c007e5e29c6 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:35.873370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 932/1866] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:35 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:35.874736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721323b7-552e-43c1-beb3-e61a02563a76 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:35.886283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721323b7-552e-43c1-beb3-e61a02563a76 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:35.949072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:35.965981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029eb9-bd1a-4b39-9b6c-911151c565a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:35.967188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029eb9-bd1a-4b39-9b6c-911151c565a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:35.967465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029eb9-bd1a-4b39-9b6c-911151c565a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:35.967732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029eb9-bd1a-4b39-9b6c-911151c565a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:35.968348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029eb9-bd1a-4b39-9b6c-911151c565a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:35.970236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029eb9-bd1a-4b39-9b6c-911151c565a6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:35.994656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 935/1867] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:35 2022] POST /identity/v3/users => generated 303 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:35.995882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3c78-a299-4690-b4f3-7c007e5e29c6 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:36.008174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3c78-a299-4690-b4f3-7c007e5e29c6 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:36.013012 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d83fb0-b668-4007-8343-f23c5b26de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:36.014204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d83fb0-b668-4007-8343-f23c5b26de9f admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:36.014981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d83fb0-b668-4007-8343-f23c5b26de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:36.014981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d83fb0-b668-4007-8343-f23c5b26de9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:36.015489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d83fb0-b668-4007-8343-f23c5b26de9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:36.017080 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d83fb0-b668-4007-8343-f23c5b26de9f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:36.029310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 933/1868] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:03:36.033680 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029eb9-bd1a-4b39-9b6c-911151c565a6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:36.044908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029eb9-bd1a-4b39-9b6c-911151c565a6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:36.049447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5655030-2a42-4914-9b6a-b0d3c11bfb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:36.050107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5655030-2a42-4914-9b6a-b0d3c11bfb25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:36.050388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5655030-2a42-4914-9b6a-b0d3c11bfb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:36.050661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5655030-2a42-4914-9b6a-b0d3c11bfb25 admin admin] PATH_INFO: `/v3/projects/89e33332ce0841718fedbcd30a13169e/users/88fb78473d124e65aab48caf42506017/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:36.068397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5655030-2a42-4914-9b6a-b0d3c11bfb25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e33332ce0841718fedbcd30a13169e, user_id=88fb78473d124e65aab48caf42506017, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:36.071220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5655030-2a42-4914-9b6a-b0d3c11bfb25 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:36.084347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 936/1869] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:36 2022] PUT /identity/v3/projects/89e33332ce0841718fedbcd30a13169e/users/88fb78473d124e65aab48caf42506017/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:36.088489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d83fb0-b668-4007-8343-f23c5b26de9f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:36.098250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d83fb0-b668-4007-8343-f23c5b26de9f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:36.143517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:36.154209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd42ea-91e5-489c-9e2e-74506f68b7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:36.154983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd42ea-91e5-489c-9e2e-74506f68b7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:36.155258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd42ea-91e5-489c-9e2e-74506f68b7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:36.155526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd42ea-91e5-489c-9e2e-74506f68b7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:36.156152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd42ea-91e5-489c-9e2e-74506f68b7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:36.157720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd42ea-91e5-489c-9e2e-74506f68b7a9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:36.167188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 934/1870] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:36.171921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5655030-2a42-4914-9b6a-b0d3c11bfb25 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:36.182506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5655030-2a42-4914-9b6a-b0d3c11bfb25 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:36.187576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfa6ee-ec0b-4eea-9009-982c5898994f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:36.188204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfa6ee-ec0b-4eea-9009-982c5898994f admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:36.188482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfa6ee-ec0b-4eea-9009-982c5898994f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:36.188751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfa6ee-ec0b-4eea-9009-982c5898994f admin admin] PATH_INFO: `/v3/projects/89e33332ce0841718fedbcd30a13169e/users/88fb78473d124e65aab48caf42506017/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:36.191298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfa6ee-ec0b-4eea-9009-982c5898994f admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e33332ce0841718fedbcd30a13169e, user_id=88fb78473d124e65aab48caf42506017, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:36.194192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfa6ee-ec0b-4eea-9009-982c5898994f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:36.205197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 937/1871] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:36 2022] PUT /identity/v3/projects/89e33332ce0841718fedbcd30a13169e/users/88fb78473d124e65aab48caf42506017/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:36.217132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcf48b-678c-44f3-b722-c6f5b613e9c4 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:36.217425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcf48b-678c-44f3-b722-c6f5b613e9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:36.217698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcf48b-678c-44f3-b722-c6f5b613e9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:36.249758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-73bcf48b-678c-44f3-b722-c6f5b613e9c4 None None] MFA Rules not processed for user `88fb78473d124e65aab48caf42506017`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:36.251346 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-73bcf48b-678c-44f3-b722-c6f5b613e9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:36.327541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 935/1872] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:36 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:36.338301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfa6ee-ec0b-4eea-9009-982c5898994f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:36.340062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b0cfa6ee-ec0b-4eea-9009-982c5898994f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:36.405173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfa6ee-ec0b-4eea-9009-982c5898994f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:36.409731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f71b4-902a-43ec-818c-c5de96853f3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:36.410353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f71b4-902a-43ec-818c-c5de96853f3f service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:36.410628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f71b4-902a-43ec-818c-c5de96853f3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:36.410901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f71b4-902a-43ec-818c-c5de96853f3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:36.420667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f71b4-902a-43ec-818c-c5de96853f3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:36.422139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f71b4-902a-43ec-818c-c5de96853f3f service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:36.431646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 938/1873] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:36 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:36.487181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcf48b-678c-44f3-b722-c6f5b613e9c4 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:36.496429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcf48b-678c-44f3-b722-c6f5b613e9c4 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:36.501329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066195a8-cdff-4f69-8045-88b28582cce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:36.501913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066195a8-cdff-4f69-8045-88b28582cce4 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:36.502187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066195a8-cdff-4f69-8045-88b28582cce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:36.502458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066195a8-cdff-4f69-8045-88b28582cce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:36.559354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066195a8-cdff-4f69-8045-88b28582cce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:36.561507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066195a8-cdff-4f69-8045-88b28582cce4 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:36.571028 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 936/1874] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:38.907571 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f71b4-902a-43ec-818c-c5de96853f3f service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:38.909060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-da4f71b4-902a-43ec-818c-c5de96853f3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:38.988235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f71b4-902a-43ec-818c-c5de96853f3f service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:38.994874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:38.997058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:38.997404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:38.998563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] PATH_INFO: `/v3/users/0bfc51f69ea74560bc3a9152f646428d` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:39.000805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfc51f69ea74560bc3a9152f646428d)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:39.003999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:39.065349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:39.078449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfc51f69ea74560bc3a9152f646428d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:39.088118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfc51f69ea74560bc3a9152f646428d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:39.105418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfc51f69ea74560bc3a9152f646428d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:39.106225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfc51f69ea74560bc3a9152f646428d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:39.108177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfc51f69ea74560bc3a9152f646428d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:39.141728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2679b-93d4-4ca8-af15-ec162c84353b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:39.142181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2679b-93d4-4ca8-af15-ec162c84353b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:39.142440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2679b-93d4-4ca8-af15-ec162c84353b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:39.143035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 939/1875] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:38 2022] DELETE /identity/v3/users/0bfc51f69ea74560bc3a9152f646428d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:39.162152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-aac2679b-93d4-4ca8-af15-ec162c84353b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:39.163381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-aac2679b-93d4-4ca8-af15-ec162c84353b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:39.254531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 937/1876] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:39.264897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:39.266398 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:39.341998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd758738-eff4-451b-87a7-20f66da185a2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:39.348272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8c603-2a69-49e0-921d-b8dfc7dfcafe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:39.349121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8c603-2a69-49e0-921d-b8dfc7dfcafe service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:39.349411 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8c603-2a69-49e0-921d-b8dfc7dfcafe service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:39.349728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8c603-2a69-49e0-921d-b8dfc7dfcafe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:39.364690 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8c603-2a69-49e0-921d-b8dfc7dfcafe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:39.366307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8c603-2a69-49e0-921d-b8dfc7dfcafe service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:39.389715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 940/1877] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:39.852820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2679b-93d4-4ca8-af15-ec162c84353b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:39.854458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-aac2679b-93d4-4ca8-af15-ec162c84353b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:39.871426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2679b-93d4-4ca8-af15-ec162c84353b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:39.883255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:39.884004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:39.885084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:39.885582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] PATH_INFO: `/v3/users/6963ee9577284a099a3120b1c03708ec` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:39.886876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6963ee9577284a099a3120b1c03708ec)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:39.889137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:39.894519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:39.948672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6963ee9577284a099a3120b1c03708ec'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:39.960566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6963ee9577284a099a3120b1c03708ec'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:39.969460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6963ee9577284a099a3120b1c03708ec'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:39.991601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6963ee9577284a099a3120b1c03708ec'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:39.992556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6963ee9577284a099a3120b1c03708ec'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:40.015852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 938/1878] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:39 2022] DELETE /identity/v3/users/6963ee9577284a099a3120b1c03708ec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:40.022204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c89ed-a44a-45d2-92ea-e2ff9d946ba3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:40.022685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c89ed-a44a-45d2-92ea-e2ff9d946ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:40.022802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c89ed-a44a-45d2-92ea-e2ff9d946ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:40.056371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ba9c89ed-a44a-45d2-92ea-e2ff9d946ba3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:40.057961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ba9c89ed-a44a-45d2-92ea-e2ff9d946ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:40.136217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 941/1879] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:40.175511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:40.177795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:40.260114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2858df-6377-4c21-b6c9-5a872d5eadd0 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:40.266506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe1e4f-6d9e-476c-9f8c-5edab1eb6c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:40.267366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe1e4f-6d9e-476c-9f8c-5edab1eb6c21 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:40.267784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe1e4f-6d9e-476c-9f8c-5edab1eb6c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:40.268288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe1e4f-6d9e-476c-9f8c-5edab1eb6c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:40.279474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe1e4f-6d9e-476c-9f8c-5edab1eb6c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:40.281132 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe1e4f-6d9e-476c-9f8c-5edab1eb6c21 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:40.293606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 939/1880] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:40.854248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c89ed-a44a-45d2-92ea-e2ff9d946ba3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:40.855841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ba9c89ed-a44a-45d2-92ea-e2ff9d946ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:40.886559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c89ed-a44a-45d2-92ea-e2ff9d946ba3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:40.893704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:40.894442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:40.894725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:40.895084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] PATH_INFO: `/v3/projects/01f9eed7eb894574af165022ff8014ff` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:40.899008 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01f9eed7eb894574af165022ff8014ff)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:40.901152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:40.986664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01f9eed7eb894574af165022ff8014ff', 'request_id': 'req-db886fde-a30e-4d5a-910b-42af29819581'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:40.998120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe1e4f-6d9e-476c-9f8c-5edab1eb6c21 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:41.002567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-64fe1e4f-6d9e-476c-9f8c-5edab1eb6c21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:41.005524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01f9eed7eb894574af165022ff8014ff', 'request_id': 'req-db886fde-a30e-4d5a-910b-42af29819581'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.007278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:41.017820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01f9eed7eb894574af165022ff8014ff', 'request_id': 'req-db886fde-a30e-4d5a-910b-42af29819581'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.035594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] The token cache is being invalidate because project 01f9eed7eb894574af165022ff8014ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:41.036663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.038184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.040419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 942/1881] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:40 2022] DELETE /identity/v3/projects/01f9eed7eb894574af165022ff8014ff => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:41.044729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:41.046620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:41.091334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe1e4f-6d9e-476c-9f8c-5edab1eb6c21 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:41.144248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db886fde-a30e-4d5a-910b-42af29819581 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:41.157241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:41.158133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:41.160540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:41.160836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] PATH_INFO: `/v3/users/bc63490de1414f9aa8f28ad2248f58ee` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:41.163878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc63490de1414f9aa8f28ad2248f58ee)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:41.168435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:41.231689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:41.232899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:41.233218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:41.233449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] PATH_INFO: `/v3/projects/6cdfbea73b3e457eb5415f5be77c2937` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:41.234724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cdfbea73b3e457eb5415f5be77c2937)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:41.236801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:41.277403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc63490de1414f9aa8f28ad2248f58ee'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.298437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc63490de1414f9aa8f28ad2248f58ee'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.317923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:41.332959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc63490de1414f9aa8f28ad2248f58ee'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.334137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc63490de1414f9aa8f28ad2248f58ee'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.335993 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc63490de1414f9aa8f28ad2248f58ee'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.361527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cdfbea73b3e457eb5415f5be77c2937', 'request_id': 'req-e60d6084-6bc5-4a46-87f0-eefb489217bd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.367148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cdfbea73b3e457eb5415f5be77c2937', 'request_id': 'req-e60d6084-6bc5-4a46-87f0-eefb489217bd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.381820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 943/1882] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:41 2022] DELETE /identity/v3/users/bc63490de1414f9aa8f28ad2248f58ee => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:41.390210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cdfbea73b3e457eb5415f5be77c2937', 'request_id': 'req-e60d6084-6bc5-4a46-87f0-eefb489217bd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.411131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] The token cache is being invalidate because project 6cdfbea73b3e457eb5415f5be77c2937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:41.413482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.414438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.417535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 940/1883] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:40 2022] DELETE /identity/v3/projects/6cdfbea73b3e457eb5415f5be77c2937 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:41.422570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:41.424072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:41.564804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fc217-fb03-4688-b080-8a92e41b5d93 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:41.574597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:41.577956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:41.583733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:41.584207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] PATH_INFO: `/v3/users/8af45ee4682448f4b4b7915b47cc2c13` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:41.586455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af45ee4682448f4b4b7915b47cc2c13)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:41.588600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:41.602856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:41.620389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:41.622028 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:41.721473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af45ee4682448f4b4b7915b47cc2c13'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.743848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af45ee4682448f4b4b7915b47cc2c13'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.771139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af45ee4682448f4b4b7915b47cc2c13'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.774944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af45ee4682448f4b4b7915b47cc2c13'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.777066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a12f063-f33f-442c-b4ab-fe25c526c117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af45ee4682448f4b4b7915b47cc2c13'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.797176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d6084-6bc5-4a46-87f0-eefb489217bd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:41.808421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 944/1884] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:41 2022] DELETE /identity/v3/users/8af45ee4682448f4b4b7915b47cc2c13 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:41.812954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:41.814274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:41.814661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:41.815098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] PATH_INFO: `/v3/users/ff04af971ce94e2782c1f6d22a548ce4` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:41.817186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff04af971ce94e2782c1f6d22a548ce4)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:41.819886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:41.935848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff04af971ce94e2782c1f6d22a548ce4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.959919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff04af971ce94e2782c1f6d22a548ce4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:41.977279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff04af971ce94e2782c1f6d22a548ce4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.011267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff04af971ce94e2782c1f6d22a548ce4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.013680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff04af971ce94e2782c1f6d22a548ce4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.030664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 941/1885] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:41 2022] DELETE /identity/v3/users/ff04af971ce94e2782c1f6d22a548ce4 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:42.035054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657f7b-3965-4a38-a755-3edeee03afad None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:42.035413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657f7b-3965-4a38-a755-3edeee03afad None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:42.036265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657f7b-3965-4a38-a755-3edeee03afad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:42.059806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-86657f7b-3965-4a38-a755-3edeee03afad None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:42.061097 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-86657f7b-3965-4a38-a755-3edeee03afad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.147960 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:42.150565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.159102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 945/1886] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:42.170007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50169dba-6013-49f8-b1a9-ece9d21ff4f0 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:42.175401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86657f7b-3965-4a38-a755-3edeee03afad None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:42.177239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-86657f7b-3965-4a38-a755-3edeee03afad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.192196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:42.192965 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:42.196367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:42.196778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] PATH_INFO: `/v3/users/f05a43407a8d487d8d2c145ac0ecbca9` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:42.199576 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05a43407a8d487d8d2c145ac0ecbca9)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:42.204071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:42.266256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:42.278774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86657f7b-3965-4a38-a755-3edeee03afad None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:42.282999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cacec-1467-46b6-b8dd-2791a5953dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:42.283756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cacec-1467-46b6-b8dd-2791a5953dff service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:42.284033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cacec-1467-46b6-b8dd-2791a5953dff service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:42.284605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cacec-1467-46b6-b8dd-2791a5953dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:42.286038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05a43407a8d487d8d2c145ac0ecbca9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.299213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05a43407a8d487d8d2c145ac0ecbca9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.299351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cacec-1467-46b6-b8dd-2791a5953dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:42.301772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cacec-1467-46b6-b8dd-2791a5953dff service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:42.312111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05a43407a8d487d8d2c145ac0ecbca9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.314295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 946/1887] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:42.315661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cacec-1467-46b6-b8dd-2791a5953dff service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:42.317264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-529cacec-1467-46b6-b8dd-2791a5953dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.333961 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05a43407a8d487d8d2c145ac0ecbca9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.334878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-770d0733-f9bd-4ba4-9b2f-df90efe2766e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05a43407a8d487d8d2c145ac0ecbca9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.342818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cacec-1467-46b6-b8dd-2791a5953dff service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:42.343588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 942/1888] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:42 2022] DELETE /identity/v3/users/f05a43407a8d487d8d2c145ac0ecbca9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:42.347965 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f505b-61b7-4685-89b9-0f27f4d221d7 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:42.348249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f505b-61b7-4685-89b9-0f27f4d221d7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:42.348526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f505b-61b7-4685-89b9-0f27f4d221d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:42.371179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ee4f505b-61b7-4685-89b9-0f27f4d221d7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:42.373221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ee4f505b-61b7-4685-89b9-0f27f4d221d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.390186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:42.446816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:42.447499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:42.447776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:42.448046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] PATH_INFO: `/v3/projects/690f7f247fef43969a37c9fde0253d18` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:42.449775 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=690f7f247fef43969a37c9fde0253d18)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:42.452631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:42.467777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 943/1889] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:42.502642 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f505b-61b7-4685-89b9-0f27f4d221d7 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:42.504308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ee4f505b-61b7-4685-89b9-0f27f4d221d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.518733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690f7f247fef43969a37c9fde0253d18', 'request_id': 'req-5ca7cb55-348d-41dd-84c4-c90b989a8456'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.528979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690f7f247fef43969a37c9fde0253d18', 'request_id': 'req-5ca7cb55-348d-41dd-84c4-c90b989a8456'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.539431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690f7f247fef43969a37c9fde0253d18', 'request_id': 'req-5ca7cb55-348d-41dd-84c4-c90b989a8456'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.571015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] The token cache is being invalidate because project 690f7f247fef43969a37c9fde0253d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:42.571491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.573027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.575441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 947/1890] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:42 2022] DELETE /identity/v3/projects/690f7f247fef43969a37c9fde0253d18 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:42.580542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:42.583536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.627610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f505b-61b7-4685-89b9-0f27f4d221d7 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:42.717305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7cb55-348d-41dd-84c4-c90b989a8456 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:42.722856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:42.723601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:42.723985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:42.724329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] PATH_INFO: `/v3/users/d1c2528bd5c448d3ad721f55525dbac1` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:42.725611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c2528bd5c448d3ad721f55525dbac1)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:42.728410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:42.760025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:42.760652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:42.760929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:42.761510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:42.764176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.785793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:42.787462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:42.799529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c2528bd5c448d3ad721f55525dbac1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.801299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 944/1891] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:42.806873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:42.806873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.814349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c2528bd5c448d3ad721f55525dbac1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.817708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:42.847229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c2528bd5c448d3ad721f55525dbac1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.851699 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf13b3-b832-4613-bfbf-c0143c39f39e service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:42.853472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c2528bd5c448d3ad721f55525dbac1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.855091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c2528bd5c448d3ad721f55525dbac1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:42.860834 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:42.862493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:42.863379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:42.863909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] PATH_INFO: `/v3/projects/3690ab2d887e42dca6d5124357e55131` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:42.865705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3690ab2d887e42dca6d5124357e55131)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:42.868523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:42.878534 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 948/1892] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:42 2022] DELETE /identity/v3/users/d1c2528bd5c448d3ad721f55525dbac1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:42.879823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:42.892127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:42.900697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:43.037530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9a45-144c-4523-a98c-ec61be0339b0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:43.051358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3690ab2d887e42dca6d5124357e55131', 'request_id': 'req-be9aafc9-e82d-47f9-aadf-399413e9b34a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.068868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3690ab2d887e42dca6d5124357e55131', 'request_id': 'req-be9aafc9-e82d-47f9-aadf-399413e9b34a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.085861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3690ab2d887e42dca6d5124357e55131', 'request_id': 'req-be9aafc9-e82d-47f9-aadf-399413e9b34a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.136429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] The token cache is being invalidate because project 3690ab2d887e42dca6d5124357e55131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:43.141552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.153040 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-be9aafc9-e82d-47f9-aadf-399413e9b34a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.161092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 945/1893] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:42 2022] DELETE /identity/v3/projects/3690ab2d887e42dca6d5124357e55131 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:43.162982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77a7aa-fb31-416a-91a6-3fb224185094 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:43.163390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77a7aa-fb31-416a-91a6-3fb224185094 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:43.163846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77a7aa-fb31-416a-91a6-3fb224185094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:43.230594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-2a77a7aa-fb31-416a-91a6-3fb224185094 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:43.232539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-2a77a7aa-fb31-416a-91a6-3fb224185094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:43.255602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:43.257834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:43.258148 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:43.258432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] PATH_INFO: `/v3/projects/691f28dd9af54c6db82585fffef29c56` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:43.262220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=691f28dd9af54c6db82585fffef29c56)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:43.264666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:43.327670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '691f28dd9af54c6db82585fffef29c56', 'request_id': 'req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.338448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '691f28dd9af54c6db82585fffef29c56', 'request_id': 'req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.349757 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '691f28dd9af54c6db82585fffef29c56', 'request_id': 'req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.365052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] The token cache is being invalidate because project 691f28dd9af54c6db82585fffef29c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:43.365606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.366429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.369103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 949/1894] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:42 2022] DELETE /identity/v3/projects/691f28dd9af54c6db82585fffef29c56 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:43.373356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:43.374992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:43.393847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:43.440752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 946/1895] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:43.443197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedaf24-16a2-4cdb-862e-a61be3b14568 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:43.443851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedaf24-16a2-4cdb-862e-a61be3b14568 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:43.443851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedaf24-16a2-4cdb-862e-a61be3b14568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:43.487083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6cedaf24-16a2-4cdb-862e-a61be3b14568 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:43.487222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851bdb1-8eed-4aa3-abf4-ab0c9a3c15b1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:43.490213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6cedaf24-16a2-4cdb-862e-a61be3b14568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:43.493875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:43.494656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:43.495123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:43.495972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] PATH_INFO: `/v3/users/88fb78473d124e65aab48caf42506017` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:43.497259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88fb78473d124e65aab48caf42506017)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:43.499976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:43.513742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 947/1896] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:43.515220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedaf24-16a2-4cdb-862e-a61be3b14568 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:43.533253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedaf24-16a2-4cdb-862e-a61be3b14568 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:43.549166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b17af0-4ffe-4138-91ce-0352a4b3a63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:43.551492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b17af0-4ffe-4138-91ce-0352a4b3a63f admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:43.551808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b17af0-4ffe-4138-91ce-0352a4b3a63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:43.552088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b17af0-4ffe-4138-91ce-0352a4b3a63f admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:43.552752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b17af0-4ffe-4138-91ce-0352a4b3a63f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:43.553050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b17af0-4ffe-4138-91ce-0352a4b3a63f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:43.555602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b17af0-4ffe-4138-91ce-0352a4b3a63f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:43.584374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 948/1897] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:03:43.586288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b17af0-4ffe-4138-91ce-0352a4b3a63f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:43.589452 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88fb78473d124e65aab48caf42506017'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.599875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b17af0-4ffe-4138-91ce-0352a4b3a63f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:43.608815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afef642-c617-4240-87df-7c25586ad9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:43.609450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afef642-c617-4240-87df-7c25586ad9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:43.609725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afef642-c617-4240-87df-7c25586ad9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:43.610035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afef642-c617-4240-87df-7c25586ad9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:43.611911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afef642-c617-4240-87df-7c25586ad9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:43.613830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afef642-c617-4240-87df-7c25586ad9ab admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:43.619109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88fb78473d124e65aab48caf42506017'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.639699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88fb78473d124e65aab48caf42506017'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.640612 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88fb78473d124e65aab48caf42506017'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.641389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88fb78473d124e65aab48caf42506017'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.666862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 950/1898] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:43 2022] DELETE /identity/v3/users/88fb78473d124e65aab48caf42506017 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:43.666862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:43.670254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:43.714001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 949/1899] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:43 2022] POST /identity/v3/projects => generated 363 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:43.715845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afef642-c617-4240-87df-7c25586ad9ab admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:43.722021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7afef642-c617-4240-87df-7c25586ad9ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:43.803985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:43.813477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009db437-f76c-4222-8c87-4394a64b06a0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:43.818845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:43.819737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:43.820043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:43.820428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] PATH_INFO: `/v3/projects/4360d19275514f4286485fea2af27414` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:43.822006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4360d19275514f4286485fea2af27414)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:43.824025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:43.826398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afef642-c617-4240-87df-7c25586ad9ab admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:43.832194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f513a9d-e650-464a-b507-d1385e2ac2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:43.833077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513a9d-e650-464a-b507-d1385e2ac2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:43.833590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513a9d-e650-464a-b507-d1385e2ac2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:43.834014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513a9d-e650-464a-b507-d1385e2ac2a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:43.837565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f513a9d-e650-464a-b507-d1385e2ac2a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:43.840761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f513a9d-e650-464a-b507-d1385e2ac2a1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:43.871167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 950/1900] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:43 2022] POST /identity/v3/users => generated 282 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:43.871167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f513a9d-e650-464a-b507-d1385e2ac2a1 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:43.871731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:43.872475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5f513a9d-e650-464a-b507-d1385e2ac2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:43.883076 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4360d19275514f4286485fea2af27414', 'request_id': 'req-4ca71a09-d878-4116-a2c2-318c7e513ca3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.888389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f513a9d-e650-464a-b507-d1385e2ac2a1 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:43.896207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4360d19275514f4286485fea2af27414', 'request_id': 'req-4ca71a09-d878-4116-a2c2-318c7e513ca3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.903480 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4360d19275514f4286485fea2af27414', 'request_id': 'req-4ca71a09-d878-4116-a2c2-318c7e513ca3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.916233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] The token cache is being invalidate because project 4360d19275514f4286485fea2af27414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:43.916522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.917188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:43.925545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 951/1901] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:43 2022] DELETE /identity/v3/projects/4360d19275514f4286485fea2af27414 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:43.925545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:43.927381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:43.962854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:43.963524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:43.963805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:43.964077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] PATH_INFO: `/v3/projects/c0e8e63b98e144fb98f04ceea3180cf5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:43.965225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0e8e63b98e144fb98f04ceea3180cf5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:43.967812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.015688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca71a09-d878-4116-a2c2-318c7e513ca3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.025035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0e8e63b98e144fb98f04ceea3180cf5', 'request_id': 'req-6606eff5-dae9-46b5-8431-6680e175326b'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.032560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0e8e63b98e144fb98f04ceea3180cf5', 'request_id': 'req-6606eff5-dae9-46b5-8431-6680e175326b'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.039949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0e8e63b98e144fb98f04ceea3180cf5', 'request_id': 'req-6606eff5-dae9-46b5-8431-6680e175326b'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.053809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] The token cache is being invalidate because project c0e8e63b98e144fb98f04ceea3180cf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:44.054103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.054840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6606eff5-dae9-46b5-8431-6680e175326b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.056411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 951/1902] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:43 2022] DELETE /identity/v3/projects/c0e8e63b98e144fb98f04ceea3180cf5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:44.059630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06916554-98c7-4712-9f1d-df67c74fdd29 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.059630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06916554-98c7-4712-9f1d-df67c74fdd29 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.059630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06916554-98c7-4712-9f1d-df67c74fdd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.077979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c99bd-9218-4d8c-9537-83cca487f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.078592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c99bd-9218-4d8c-9537-83cca487f35e admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.078871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c99bd-9218-4d8c-9537-83cca487f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.079098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-06916554-98c7-4712-9f1d-df67c74fdd29 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:44.079219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c99bd-9218-4d8c-9537-83cca487f35e admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.079797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c99bd-9218-4d8c-9537-83cca487f35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.080283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-06916554-98c7-4712-9f1d-df67c74fdd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.081397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c99bd-9218-4d8c-9537-83cca487f35e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.095216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 952/1903] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:44.095390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 952/1904] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:44.095390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c99bd-9218-4d8c-9537-83cca487f35e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.096291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6b7c99bd-9218-4d8c-9537-83cca487f35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.096531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06916554-98c7-4712-9f1d-df67c74fdd29 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.097800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-06916554-98c7-4712-9f1d-df67c74fdd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.108967 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c99bd-9218-4d8c-9537-83cca487f35e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.109519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06916554-98c7-4712-9f1d-df67c74fdd29 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.114030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.114637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.114782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.114905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.115278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] PATH_INFO: `/v3/projects/89e33332ce0841718fedbcd30a13169e` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.115450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.115831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.116107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] PATH_INFO: `/v3/users/9ac3492e69ae418b8d9a161f6472e340` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.116367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e33332ce0841718fedbcd30a13169e)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.117241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac3492e69ae418b8d9a161f6472e340)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.118309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.119220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.171186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e33332ce0841718fedbcd30a13169e', 'request_id': 'req-13377ec2-39af-4ff2-a678-4221ccd8d17c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.171907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac3492e69ae418b8d9a161f6472e340'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.179410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e33332ce0841718fedbcd30a13169e', 'request_id': 'req-13377ec2-39af-4ff2-a678-4221ccd8d17c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.182801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac3492e69ae418b8d9a161f6472e340'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.186826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=80,keystone:INSERT=5,keystone:DELETE=15 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:44.187847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e33332ce0841718fedbcd30a13169e', 'request_id': 'req-13377ec2-39af-4ff2-a678-4221ccd8d17c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.189586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:44.196205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac3492e69ae418b8d9a161f6472e340'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.203836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] The token cache is being invalidate because project 89e33332ce0841718fedbcd30a13169e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:44.204133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.205444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.208121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 953/1905] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:44 2022] DELETE /identity/v3/projects/89e33332ce0841718fedbcd30a13169e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:44.212725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac3492e69ae418b8d9a161f6472e340'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.213600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.213745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-737d8d1a-7b99-472e-b211-841b355ad205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac3492e69ae418b8d9a161f6472e340'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.215557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.219993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 953/1906] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:44 2022] DELETE /identity/v3/users/9ac3492e69ae418b8d9a161f6472e340 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:44.221208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779667-cb08-41c2-b678-a04f4909f592 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.221490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779667-cb08-41c2-b678-a04f4909f592 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.221762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779667-cb08-41c2-b678-a04f4909f592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.246659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-61779667-cb08-41c2-b678-a04f4909f592 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:44.248494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-61779667-cb08-41c2-b678-a04f4909f592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.311502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13377ec2-39af-4ff2-a678-4221ccd8d17c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.321489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 954/1907] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:44.322885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bce2c-9785-490f-b1b5-59f0e737ce10 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.323286 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bce2c-9785-490f-b1b5-59f0e737ce10 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.323565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bce2c-9785-490f-b1b5-59f0e737ce10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.352767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-3b2bce2c-9785-490f-b1b5-59f0e737ce10 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:44.354019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3b2bce2c-9785-490f-b1b5-59f0e737ce10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.363192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 955/1908] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:44.364066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bce2c-9785-490f-b1b5-59f0e737ce10 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.374381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bce2c-9785-490f-b1b5-59f0e737ce10 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.379872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d9631-bb57-4dbd-848e-19cdb4e8d90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.380488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d9631-bb57-4dbd-848e-19cdb4e8d90c admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.380769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d9631-bb57-4dbd-848e-19cdb4e8d90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.381040 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d9631-bb57-4dbd-848e-19cdb4e8d90c admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.381755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d9631-bb57-4dbd-848e-19cdb4e8d90c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:44.382210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d9631-bb57-4dbd-848e-19cdb4e8d90c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.383693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d9631-bb57-4dbd-848e-19cdb4e8d90c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.404876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 956/1909] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:44.404876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383d867-4ba5-46c4-91ea-50d94a6fe649 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.404876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383d867-4ba5-46c4-91ea-50d94a6fe649 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.404876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383d867-4ba5-46c4-91ea-50d94a6fe649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.424139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36632f-73f6-4fba-948e-b883fa0fa3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.424806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36632f-73f6-4fba-948e-b883fa0fa3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.425308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36632f-73f6-4fba-948e-b883fa0fa3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.425590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36632f-73f6-4fba-948e-b883fa0fa3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.426381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36632f-73f6-4fba-948e-b883fa0fa3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.428050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36632f-73f6-4fba-948e-b883fa0fa3c2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.444520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 954/1910] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:44.445587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36632f-73f6-4fba-948e-b883fa0fa3c2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.446776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6383d867-4ba5-46c4-91ea-50d94a6fe649 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:44.448253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6383d867-4ba5-46c4-91ea-50d94a6fe649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.458662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 957/1911] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:44.462465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383d867-4ba5-46c4-91ea-50d94a6fe649 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.462465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6383d867-4ba5-46c4-91ea-50d94a6fe649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.463793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36632f-73f6-4fba-948e-b883fa0fa3c2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.469911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d50216-11c7-4a21-b4d1-048496c8e0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.470582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d50216-11c7-4a21-b4d1-048496c8e0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.470856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d50216-11c7-4a21-b4d1-048496c8e0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.471152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d50216-11c7-4a21-b4d1-048496c8e0f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.471832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d50216-11c7-4a21-b4d1-048496c8e0f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.474086 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d50216-11c7-4a21-b4d1-048496c8e0f2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.492579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383d867-4ba5-46c4-91ea-50d94a6fe649 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.502267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7362a-5638-4d75-9b42-d004d5d78c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.503476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7362a-5638-4d75-9b42-d004d5d78c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.503769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7362a-5638-4d75-9b42-d004d5d78c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.504045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7362a-5638-4d75-9b42-d004d5d78c0a admin admin] PATH_INFO: `/v3/projects/ac434880f080463cad7365dda60a1a25/users/ae143e8b59db4fc481d64fe359b32bb6/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.531191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 955/1912] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:44 2022] POST /identity/v3/projects => generated 381 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:44.532477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d50216-11c7-4a21-b4d1-048496c8e0f2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.536811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7362a-5638-4d75-9b42-d004d5d78c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac434880f080463cad7365dda60a1a25, user_id=ae143e8b59db4fc481d64fe359b32bb6, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.540572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7362a-5638-4d75-9b42-d004d5d78c0a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.565639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 958/1913] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:44 2022] PUT /identity/v3/projects/ac434880f080463cad7365dda60a1a25/users/ae143e8b59db4fc481d64fe359b32bb6/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:44.566456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7362a-5638-4d75-9b42-d004d5d78c0a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.639591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d50216-11c7-4a21-b4d1-048496c8e0f2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.655724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:44.677339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7362a-5638-4d75-9b42-d004d5d78c0a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.689494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2dbf0-6af9-4506-9d38-72df27731d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.690481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2dbf0-6af9-4506-9d38-72df27731d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.690876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2dbf0-6af9-4506-9d38-72df27731d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.691254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2dbf0-6af9-4506-9d38-72df27731d58 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.692129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2dbf0-6af9-4506-9d38-72df27731d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.694662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2dbf0-6af9-4506-9d38-72df27731d58 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.728240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 959/1914] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:44 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:44.729558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557d059-67d7-46a6-9cff-9061d14b089d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.729765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557d059-67d7-46a6-9cff-9061d14b089d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.730089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557d059-67d7-46a6-9cff-9061d14b089d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.744368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.744974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.745254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.745525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.746175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:44.746507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.748841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.767325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:44.768606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 956/1915] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:44.769730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.771882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d557d059-67d7-46a6-9cff-9061d14b089d None None] MFA Rules not processed for user `ae143e8b59db4fc481d64fe359b32bb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:44.772002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.773120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d557d059-67d7-46a6-9cff-9061d14b089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.791785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c16a7-b291-481e-8707-456cf9b1a4a9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.799800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.800446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.800763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.800993 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] PATH_INFO: `/v3/projects/7765f0bee2b2436383c1805f87af3e33` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.802146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7765f0bee2b2436383c1805f87af3e33)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.804251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.853936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 960/1916] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:44.858174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557d059-67d7-46a6-9cff-9061d14b089d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.868305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557d059-67d7-46a6-9cff-9061d14b089d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.896988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7765f0bee2b2436383c1805f87af3e33', 'request_id': 'req-3d5ca249-22ce-4908-b2c1-2f4de78857a7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.909870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7765f0bee2b2436383c1805f87af3e33', 'request_id': 'req-3d5ca249-22ce-4908-b2c1-2f4de78857a7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.920520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7765f0bee2b2436383c1805f87af3e33', 'request_id': 'req-3d5ca249-22ce-4908-b2c1-2f4de78857a7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.938640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] The token cache is being invalidate because project 7765f0bee2b2436383c1805f87af3e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:44.938936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.939312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4131d38-1e08-4d5b-89b9-dc699b6be353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.939842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:44.940159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4131d38-1e08-4d5b-89b9-dc699b6be353 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.940555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4131d38-1e08-4d5b-89b9-dc699b6be353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.941520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4131d38-1e08-4d5b-89b9-dc699b6be353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.941632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4131d38-1e08-4d5b-89b9-dc699b6be353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.941745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 957/1917] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:44 2022] DELETE /identity/v3/projects/7765f0bee2b2436383c1805f87af3e33 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:44.942763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.943871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4131d38-1e08-4d5b-89b9-dc699b6be353 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.944646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.958135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 961/1918] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:44.959835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ca249-22ce-4908-b2c1-2f4de78857a7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.962977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4131d38-1e08-4d5b-89b9-dc699b6be353 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:44.965334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f4131d38-1e08-4d5b-89b9-dc699b6be353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:44.966440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad254fbb-f9fc-49d6-b871-906a0eff82fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.967177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad254fbb-f9fc-49d6-b871-906a0eff82fb admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.967571 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad254fbb-f9fc-49d6-b871-906a0eff82fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.967916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad254fbb-f9fc-49d6-b871-906a0eff82fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.968614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad254fbb-f9fc-49d6-b871-906a0eff82fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.970616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad254fbb-f9fc-49d6-b871-906a0eff82fb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:44.981488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4131d38-1e08-4d5b-89b9-dc699b6be353 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:44.988616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb05dd-14fd-42fe-baf0-7ecdf7b0470c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:44.989401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb05dd-14fd-42fe-baf0-7ecdf7b0470c admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:44.989506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb05dd-14fd-42fe-baf0-7ecdf7b0470c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:44.989741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb05dd-14fd-42fe-baf0-7ecdf7b0470c admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:44.990380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb05dd-14fd-42fe-baf0-7ecdf7b0470c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:44.992271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb05dd-14fd-42fe-baf0-7ecdf7b0470c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.005338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 958/1919] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:44 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:45.006684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f1cc-51ee-4bdb-995d-8d06c972b7a4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.007205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f1cc-51ee-4bdb-995d-8d06c972b7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.007584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f1cc-51ee-4bdb-995d-8d06c972b7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.029634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3d13f1cc-51ee-4bdb-995d-8d06c972b7a4 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:45.030825 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3d13f1cc-51ee-4bdb-995d-8d06c972b7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:45.031429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 962/1920] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:44 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:45.035233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb05dd-14fd-42fe-baf0-7ecdf7b0470c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.037808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f0bb05dd-14fd-42fe-baf0-7ecdf7b0470c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:45.101001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 959/1921] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:45.101776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13f1cc-51ee-4bdb-995d-8d06c972b7a4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.114897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13f1cc-51ee-4bdb-995d-8d06c972b7a4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.121210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0b69c-5370-4ec3-b4e1-abfbbe2bbc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.122155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0b69c-5370-4ec3-b4e1-abfbbe2bbc5f admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.123098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0b69c-5370-4ec3-b4e1-abfbbe2bbc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.123609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0b69c-5370-4ec3-b4e1-abfbbe2bbc5f admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.124248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0b69c-5370-4ec3-b4e1-abfbbe2bbc5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.126528 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0b69c-5370-4ec3-b4e1-abfbbe2bbc5f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.136124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb05dd-14fd-42fe-baf0-7ecdf7b0470c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.138199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:45.143175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a386aa-3b3c-4646-b47a-82d4d9a3871c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.143815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a386aa-3b3c-4646-b47a-82d4d9a3871c admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.144097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a386aa-3b3c-4646-b47a-82d4d9a3871c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.144416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a386aa-3b3c-4646-b47a-82d4d9a3871c admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.145269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a386aa-3b3c-4646-b47a-82d4d9a3871c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.147512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a386aa-3b3c-4646-b47a-82d4d9a3871c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.155449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 960/1922] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/users => generated 292 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:45.156539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0b69c-5370-4ec3-b4e1-abfbbe2bbc5f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.162353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 963/1923] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:45.162353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145a564-05b6-4e15-9b7c-496e0b20e132 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.162353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145a564-05b6-4e15-9b7c-496e0b20e132 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.162353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145a564-05b6-4e15-9b7c-496e0b20e132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.167743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0b69c-5370-4ec3-b4e1-abfbbe2bbc5f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.174483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624b3d-5fd6-4db6-ae52-2e2790da3d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.174483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624b3d-5fd6-4db6-ae52-2e2790da3d8a admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.175284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624b3d-5fd6-4db6-ae52-2e2790da3d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.175727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624b3d-5fd6-4db6-ae52-2e2790da3d8a admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.176472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b624b3d-5fd6-4db6-ae52-2e2790da3d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.178487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b624b3d-5fd6-4db6-ae52-2e2790da3d8a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.185751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a145a564-05b6-4e15-9b7c-496e0b20e132 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:45.187159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a145a564-05b6-4e15-9b7c-496e0b20e132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:45.201464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 964/1924] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:45.201464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145a564-05b6-4e15-9b7c-496e0b20e132 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.206660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 961/1925] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/users => generated 288 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:45.208025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624b3d-5fd6-4db6-ae52-2e2790da3d8a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.211931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145a564-05b6-4e15-9b7c-496e0b20e132 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.218221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1effc69-6a86-4338-9e7a-46b4a90d762e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.218902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1effc69-6a86-4338-9e7a-46b4a90d762e admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.219225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1effc69-6a86-4338-9e7a-46b4a90d762e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.219490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1effc69-6a86-4338-9e7a-46b4a90d762e admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.220181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1effc69-6a86-4338-9e7a-46b4a90d762e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.220291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624b3d-5fd6-4db6-ae52-2e2790da3d8a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.221802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1effc69-6a86-4338-9e7a-46b4a90d762e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.223640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:45.229787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8535c-552b-4757-b1a7-9fd54e1ddd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.232570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8535c-552b-4757-b1a7-9fd54e1ddd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.233002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8535c-552b-4757-b1a7-9fd54e1ddd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.233601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8535c-552b-4757-b1a7-9fd54e1ddd2f admin admin] PATH_INFO: `/v3/projects/72bffafa3da048ef84af4afdb61d9b5c/users/32a2d302049c4552928ce2ca0167d538/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.242693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 965/1926] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:45.243654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1effc69-6a86-4338-9e7a-46b4a90d762e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.252824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8535c-552b-4757-b1a7-9fd54e1ddd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=72bffafa3da048ef84af4afdb61d9b5c, user_id=32a2d302049c4552928ce2ca0167d538, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.255934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1effc69-6a86-4338-9e7a-46b4a90d762e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.256370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8535c-552b-4757-b1a7-9fd54e1ddd2f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.262133 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4dbc7-2dc3-4e59-ad8b-924ba3ced1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.262934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4dbc7-2dc3-4e59-ad8b-924ba3ced1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.263716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4dbc7-2dc3-4e59-ad8b-924ba3ced1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.263832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4dbc7-2dc3-4e59-ad8b-924ba3ced1ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.264575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4dbc7-2dc3-4e59-ad8b-924ba3ced1ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:45.265032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4dbc7-2dc3-4e59-ad8b-924ba3ced1ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.266880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4dbc7-2dc3-4e59-ad8b-924ba3ced1ea admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.284563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 962/1927] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:45 2022] PUT /identity/v3/projects/72bffafa3da048ef84af4afdb61d9b5c/users/32a2d302049c4552928ce2ca0167d538/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:45.284563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8535c-552b-4757-b1a7-9fd54e1ddd2f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.289637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 966/1928] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:45.290793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4dbc7-2dc3-4e59-ad8b-924ba3ced1ea admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.303080 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4dbc7-2dc3-4e59-ad8b-924ba3ced1ea admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.353185 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8535c-552b-4757-b1a7-9fd54e1ddd2f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.357996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec4ce4-bdaf-49fa-b03c-5e93db33d0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.358603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec4ce4-bdaf-49fa-b03c-5e93db33d0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.358899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec4ce4-bdaf-49fa-b03c-5e93db33d0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.359199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec4ce4-bdaf-49fa-b03c-5e93db33d0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.359844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec4ce4-bdaf-49fa-b03c-5e93db33d0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.361431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec4ce4-bdaf-49fa-b03c-5e93db33d0d3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.371368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea7717-ecfc-4dfb-b918-cc4622a0d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.371606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 963/1929] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:45.372845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea7717-ecfc-4dfb-b918-cc4622a0d4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.373195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea7717-ecfc-4dfb-b918-cc4622a0d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.373314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055c995-b0f2-4f2a-a398-4141c61471e9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.373570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055c995-b0f2-4f2a-a398-4141c61471e9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.373667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea7717-ecfc-4dfb-b918-cc4622a0d4b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.373897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055c995-b0f2-4f2a-a398-4141c61471e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.374382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea7717-ecfc-4dfb-b918-cc4622a0d4b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.376324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea7717-ecfc-4dfb-b918-cc4622a0d4b1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.387228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 967/1930] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:45.388538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea7717-ecfc-4dfb-b918-cc4622a0d4b1 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.398827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea7717-ecfc-4dfb-b918-cc4622a0d4b1 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.412799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7055c995-b0f2-4f2a-a398-4141c61471e9 None None] MFA Rules not processed for user `32a2d302049c4552928ce2ca0167d538`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:45.414358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7055c995-b0f2-4f2a-a398-4141c61471e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:45.414465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac92aec-f348-434e-801d-43cc3ef25366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.415023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac92aec-f348-434e-801d-43cc3ef25366 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.415751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac92aec-f348-434e-801d-43cc3ef25366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.416086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac92aec-f348-434e-801d-43cc3ef25366 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.416673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac92aec-f348-434e-801d-43cc3ef25366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.418756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac92aec-f348-434e-801d-43cc3ef25366 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.450188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 968/1931] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:45.454341 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac92aec-f348-434e-801d-43cc3ef25366 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.504264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 964/1932] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:45.505218 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055c995-b0f2-4f2a-a398-4141c61471e9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.519398 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055c995-b0f2-4f2a-a398-4141c61471e9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.524881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac92aec-f348-434e-801d-43cc3ef25366 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.526219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3968e7-f9fc-4109-af3b-b2ba06474231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.527082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3968e7-f9fc-4109-af3b-b2ba06474231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.527491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3968e7-f9fc-4109-af3b-b2ba06474231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.527919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3968e7-f9fc-4109-af3b-b2ba06474231 admin admin] PATH_INFO: `/v3/projects/3a860dde44aa4e7f86ddd57055bece30/users/9488b5c821a244ddb49407cb2f53c651/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.530097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0648ea-905f-4296-ab3d-e4a0a9c9601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.530755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0648ea-905f-4296-ab3d-e4a0a9c9601c admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.531063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0648ea-905f-4296-ab3d-e4a0a9c9601c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.531340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0648ea-905f-4296-ab3d-e4a0a9c9601c admin admin] PATH_INFO: `/v3/projects/e9041a2d35794bb3b3e920d9df519ceb/users/8d29acb533a64b8794158435943da830/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.550629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0648ea-905f-4296-ab3d-e4a0a9c9601c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9041a2d35794bb3b3e920d9df519ceb, user_id=8d29acb533a64b8794158435943da830, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.552442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3968e7-f9fc-4109-af3b-b2ba06474231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a860dde44aa4e7f86ddd57055bece30, user_id=9488b5c821a244ddb49407cb2f53c651, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.553567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0648ea-905f-4296-ab3d-e4a0a9c9601c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.555798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3968e7-f9fc-4109-af3b-b2ba06474231 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.569434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 969/1933] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:45 2022] PUT /identity/v3/projects/e9041a2d35794bb3b3e920d9df519ceb/users/8d29acb533a64b8794158435943da830/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:45.570594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0648ea-905f-4296-ab3d-e4a0a9c9601c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.578653 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 965/1934] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:45 2022] PUT /identity/v3/projects/3a860dde44aa4e7f86ddd57055bece30/users/9488b5c821a244ddb49407cb2f53c651/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:45.579984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436901ba-10c5-4b71-bd83-8a8e8d66d75a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.580272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436901ba-10c5-4b71-bd83-8a8e8d66d75a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.580549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436901ba-10c5-4b71-bd83-8a8e8d66d75a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.581963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 966/1935] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 10:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:03:45.582653 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436901ba-10c5-4b71-bd83-8a8e8d66d75a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.631485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:45.643638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:45.683667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0648ea-905f-4296-ab3d-e4a0a9c9601c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.718178 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436901ba-10c5-4b71-bd83-8a8e8d66d75a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.724547 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45ff10-f974-409d-989f-0a0006928bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.725161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45ff10-f974-409d-989f-0a0006928bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.725447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45ff10-f974-409d-989f-0a0006928bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.725724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45ff10-f974-409d-989f-0a0006928bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.726809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45ff10-f974-409d-989f-0a0006928bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.728407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45ff10-f974-409d-989f-0a0006928bf4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.775103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 967/1936] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:45.776014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45ff10-f974-409d-989f-0a0006928bf4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.785743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e2a87-5ef8-45aa-9584-88c6063707f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.786622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e2a87-5ef8-45aa-9584-88c6063707f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.786992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e2a87-5ef8-45aa-9584-88c6063707f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.787307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e2a87-5ef8-45aa-9584-88c6063707f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.787999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e2a87-5ef8-45aa-9584-88c6063707f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.790092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e2a87-5ef8-45aa-9584-88c6063707f5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.822258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 970/1937] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:45.823876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda05b0-8fb1-4c7d-977b-f4cfd38b59a9 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.824161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda05b0-8fb1-4c7d-977b-f4cfd38b59a9 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.824433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda05b0-8fb1-4c7d-977b-f4cfd38b59a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.852170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45ff10-f974-409d-989f-0a0006928bf4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.855791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-9cda05b0-8fb1-4c7d-977b-f4cfd38b59a9 None None] MFA Rules not processed for user `9488b5c821a244ddb49407cb2f53c651`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:45.858340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9cda05b0-8fb1-4c7d-977b-f4cfd38b59a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:45.862530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45cad2-741c-4c48-ab0f-6080b8d0fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.863172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45cad2-741c-4c48-ab0f-6080b8d0fdd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.863450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45cad2-741c-4c48-ab0f-6080b8d0fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.863721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45cad2-741c-4c48-ab0f-6080b8d0fdd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.864368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45cad2-741c-4c48-ab0f-6080b8d0fdd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.865957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45cad2-741c-4c48-ab0f-6080b8d0fdd3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.884606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 968/1938] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:45.884606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9236e-3d42-41be-9d43-4abc1c1b0f90 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.884606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9236e-3d42-41be-9d43-4abc1c1b0f90 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.884606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9236e-3d42-41be-9d43-4abc1c1b0f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.906253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fdc9236e-3d42-41be-9d43-4abc1c1b0f90 None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:45.907988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fdc9236e-3d42-41be-9d43-4abc1c1b0f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:45.961751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 971/1939] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:45.961751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda05b0-8fb1-4c7d-977b-f4cfd38b59a9 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:45.976769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda05b0-8fb1-4c7d-977b-f4cfd38b59a9 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:45.982997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735a508-c617-430e-8de9-41bbb87f9c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:45.983751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735a508-c617-430e-8de9-41bbb87f9c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:45.984022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735a508-c617-430e-8de9-41bbb87f9c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:45.984295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735a508-c617-430e-8de9-41bbb87f9c32 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:45.985038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735a508-c617-430e-8de9-41bbb87f9c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:45.987393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735a508-c617-430e-8de9-41bbb87f9c32 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:45.990517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 969/1940] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:45.991744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9236e-3d42-41be-9d43-4abc1c1b0f90 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.003920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9236e-3d42-41be-9d43-4abc1c1b0f90 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.008844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad41c1-09de-4872-8b73-08ff8c50ed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.009721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad41c1-09de-4872-8b73-08ff8c50ed81 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.010234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad41c1-09de-4872-8b73-08ff8c50ed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.010706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad41c1-09de-4872-8b73-08ff8c50ed81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.011605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad41c1-09de-4872-8b73-08ff8c50ed81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.013556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad41c1-09de-4872-8b73-08ff8c50ed81 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.029377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 972/1941] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:45 2022] POST /identity/v3/users => generated 297 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:46.030307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735a508-c617-430e-8de9-41bbb87f9c32 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.031123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 970/1942] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:46.032089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad41c1-09de-4872-8b73-08ff8c50ed81 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.046606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735a508-c617-430e-8de9-41bbb87f9c32 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.049185 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad41c1-09de-4872-8b73-08ff8c50ed81 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.051816 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9415567-c62f-42c1-a859-f36a0deaeaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.052540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9415567-c62f-42c1-a859-f36a0deaeaa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.052756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9415567-c62f-42c1-a859-f36a0deaeaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.053027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9415567-c62f-42c1-a859-f36a0deaeaa5 admin admin] PATH_INFO: `/v3/projects/e9041a2d35794bb3b3e920d9df519ceb/users/8d29acb533a64b8794158435943da830/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.055722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9415567-c62f-42c1-a859-f36a0deaeaa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9041a2d35794bb3b3e920d9df519ceb, user_id=8d29acb533a64b8794158435943da830, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.055838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9e238-499c-4527-9182-2ca6d506e341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.056554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9e238-499c-4527-9182-2ca6d506e341 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.057476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9e238-499c-4527-9182-2ca6d506e341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.057763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9e238-499c-4527-9182-2ca6d506e341 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.058457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9e238-499c-4527-9182-2ca6d506e341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.058644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9415567-c62f-42c1-a859-f36a0deaeaa5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.061719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9e238-499c-4527-9182-2ca6d506e341 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.081623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 973/1943] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:46 2022] PUT /identity/v3/projects/e9041a2d35794bb3b3e920d9df519ceb/users/8d29acb533a64b8794158435943da830/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:46.083267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:46.084149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9415567-c62f-42c1-a859-f36a0deaeaa5 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.116177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 971/1944] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:46 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:46.117687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9e238-499c-4527-9182-2ca6d506e341 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.166475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:46.209208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9415567-c62f-42c1-a859-f36a0deaeaa5 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.250540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9e238-499c-4527-9182-2ca6d506e341 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.265475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d02d9-6937-4435-8f48-6882f0aa7f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.266724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d02d9-6937-4435-8f48-6882f0aa7f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.267042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d02d9-6937-4435-8f48-6882f0aa7f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.267317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d02d9-6937-4435-8f48-6882f0aa7f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.268094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d02d9-6937-4435-8f48-6882f0aa7f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.269893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d02d9-6937-4435-8f48-6882f0aa7f82 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.288732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 972/1945] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:46.292969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8487d-f197-4197-a3bb-d493a58c24ec None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.293234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8487d-f197-4197-a3bb-d493a58c24ec None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.293484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8487d-f197-4197-a3bb-d493a58c24ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.319231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d69c0-865c-4d55-8b82-23900f6511db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.320182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d69c0-865c-4d55-8b82-23900f6511db admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.320618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d69c0-865c-4d55-8b82-23900f6511db admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.321034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d69c0-865c-4d55-8b82-23900f6511db admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.322032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d69c0-865c-4d55-8b82-23900f6511db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.324632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d69c0-865c-4d55-8b82-23900f6511db admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.329950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-73a8487d-f197-4197-a3bb-d493a58c24ec None None] MFA Rules not processed for user `8d29acb533a64b8794158435943da830`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:46.331277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-73a8487d-f197-4197-a3bb-d493a58c24ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:46.353111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 974/1946] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:46.354037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d69c0-865c-4d55-8b82-23900f6511db admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.355944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d59d69c0-865c-4d55-8b82-23900f6511db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:46.455845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 973/1947] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:46.456512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8487d-f197-4197-a3bb-d493a58c24ec None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.486887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8487d-f197-4197-a3bb-d493a58c24ec None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.497721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ae67-7ab2-4f94-a07f-881be99396a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.498336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ae67-7ab2-4f94-a07f-881be99396a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.498723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ae67-7ab2-4f94-a07f-881be99396a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.498923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ae67-7ab2-4f94-a07f-881be99396a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.499532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ae67-7ab2-4f94-a07f-881be99396a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.502517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ae67-7ab2-4f94-a07f-881be99396a3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.533336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d69c0-865c-4d55-8b82-23900f6511db admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.542288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdd683-6d33-4552-9451-10cc1303b61c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.542917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdd683-6d33-4552-9451-10cc1303b61c service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.543224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdd683-6d33-4552-9451-10cc1303b61c service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.543503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdd683-6d33-4552-9451-10cc1303b61c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.554937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdd683-6d33-4552-9451-10cc1303b61c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.555966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 974/1948] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:46 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:46.557603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdd683-6d33-4552-9451-10cc1303b61c service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.559441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ae67-7ab2-4f94-a07f-881be99396a3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.569418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ae67-7ab2-4f94-a07f-881be99396a3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.570141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 975/1949] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:46.570973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdd683-6d33-4552-9451-10cc1303b61c service nova] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.582161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdd683-6d33-4552-9451-10cc1303b61c service nova] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.586756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d636a73-654a-425b-a411-ff20eb9b09ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.587046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259b29e-84f3-4229-ae48-0af5bb7934b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.587430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d636a73-654a-425b-a411-ff20eb9b09ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.587747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259b29e-84f3-4229-ae48-0af5bb7934b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.587875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d636a73-654a-425b-a411-ff20eb9b09ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.587973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259b29e-84f3-4229-ae48-0af5bb7934b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.588059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d636a73-654a-425b-a411-ff20eb9b09ab admin admin] PATH_INFO: `/v3/projects/47f5acbb2176422c91f24759ee8ee931/users/3911e03be7dd449c8b2a45273ec0db39/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.588215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259b29e-84f3-4229-ae48-0af5bb7934b8 admin admin] PATH_INFO: `/v3/projects/01fdc9650a53413f9e9a15f96cabf43b/users/47fa663054ef4772a379e885844d5e30/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.605512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259b29e-84f3-4229-ae48-0af5bb7934b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01fdc9650a53413f9e9a15f96cabf43b, user_id=47fa663054ef4772a379e885844d5e30, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.605757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d636a73-654a-425b-a411-ff20eb9b09ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f5acbb2176422c91f24759ee8ee931, user_id=3911e03be7dd449c8b2a45273ec0db39, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.608205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259b29e-84f3-4229-ae48-0af5bb7934b8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.609060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d636a73-654a-425b-a411-ff20eb9b09ab admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.623090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 975/1950] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:46 2022] PUT /identity/v3/projects/01fdc9650a53413f9e9a15f96cabf43b/users/47fa663054ef4772a379e885844d5e30/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:46.627276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259b29e-84f3-4229-ae48-0af5bb7934b8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.627931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 976/1951] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:46 2022] PUT /identity/v3/projects/47f5acbb2176422c91f24759ee8ee931/users/3911e03be7dd449c8b2a45273ec0db39/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:46.629013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d636a73-654a-425b-a411-ff20eb9b09ab admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.629517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5259b29e-84f3-4229-ae48-0af5bb7934b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:46.676175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:46.678243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:46.697650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259b29e-84f3-4229-ae48-0af5bb7934b8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.703220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d636a73-654a-425b-a411-ff20eb9b09ab admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.708560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01807a-4def-40c3-bd44-d88bbb5fb12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.709317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01807a-4def-40c3-bd44-d88bbb5fb12a admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.709601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01807a-4def-40c3-bd44-d88bbb5fb12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.709872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01807a-4def-40c3-bd44-d88bbb5fb12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.710521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01807a-4def-40c3-bd44-d88bbb5fb12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.712144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01807a-4def-40c3-bd44-d88bbb5fb12a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.722880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 977/1952] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:46.724372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01807a-4def-40c3-bd44-d88bbb5fb12a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.736494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01807a-4def-40c3-bd44-d88bbb5fb12a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.742325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12c9bb-0337-4476-9bb5-442936fae98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.743357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12c9bb-0337-4476-9bb5-442936fae98d admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.743821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12c9bb-0337-4476-9bb5-442936fae98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.744509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12c9bb-0337-4476-9bb5-442936fae98d admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.745461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12c9bb-0337-4476-9bb5-442936fae98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.747806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12c9bb-0337-4476-9bb5-442936fae98d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.758940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024eda1-115e-46ff-b51c-316bce09ea3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.759573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024eda1-115e-46ff-b51c-316bce09ea3d service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.759985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024eda1-115e-46ff-b51c-316bce09ea3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.760292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 978/1953] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:46.761756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f811b5-2e85-48d2-97f3-9223b9ddcfe9 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.762048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f811b5-2e85-48d2-97f3-9223b9ddcfe9 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.762342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f811b5-2e85-48d2-97f3-9223b9ddcfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.764744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024eda1-115e-46ff-b51c-316bce09ea3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.793469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-44f811b5-2e85-48d2-97f3-9223b9ddcfe9 None None] MFA Rules not processed for user `47fa663054ef4772a379e885844d5e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:46.794712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-44f811b5-2e85-48d2-97f3-9223b9ddcfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:46.830663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024eda1-115e-46ff-b51c-316bce09ea3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.832264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024eda1-115e-46ff-b51c-316bce09ea3d service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.843738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 976/1954] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:46.844658 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024eda1-115e-46ff-b51c-316bce09ea3d service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.860512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024eda1-115e-46ff-b51c-316bce09ea3d service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.867856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed0afe-c03a-4033-8a99-ab21e184be2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.868525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed0afe-c03a-4033-8a99-ab21e184be2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.868806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed0afe-c03a-4033-8a99-ab21e184be2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.869081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed0afe-c03a-4033-8a99-ab21e184be2a admin admin] PATH_INFO: `/v3/projects/e50b6fbedfd74b79b4e4fb0f099e24a0/users/2b1a8cb417184b43bb446655a681736a/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.885618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 979/1955] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:46.886844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed0afe-c03a-4033-8a99-ab21e184be2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50b6fbedfd74b79b4e4fb0f099e24a0, user_id=2b1a8cb417184b43bb446655a681736a, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:46.891954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f811b5-2e85-48d2-97f3-9223b9ddcfe9 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.893217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed0afe-c03a-4033-8a99-ab21e184be2a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:46.913382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f811b5-2e85-48d2-97f3-9223b9ddcfe9 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:46.914426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 977/1956] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:46 2022] PUT /identity/v3/projects/e50b6fbedfd74b79b4e4fb0f099e24a0/users/2b1a8cb417184b43bb446655a681736a/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:46.916914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed0afe-c03a-4033-8a99-ab21e184be2a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:46.919471 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-99ed0afe-c03a-4033-8a99-ab21e184be2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:46.996070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83551c4-20f8-4fc3-b596-a30f43335df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:46.996709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83551c4-20f8-4fc3-b596-a30f43335df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:46.996837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83551c4-20f8-4fc3-b596-a30f43335df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:46.997100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83551c4-20f8-4fc3-b596-a30f43335df4 admin admin] PATH_INFO: `/v3/projects/47f5acbb2176422c91f24759ee8ee931/users/3911e03be7dd449c8b2a45273ec0db39/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:46.999906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83551c4-20f8-4fc3-b596-a30f43335df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f5acbb2176422c91f24759ee8ee931, user_id=3911e03be7dd449c8b2a45273ec0db39, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:47.002870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83551c4-20f8-4fc3-b596-a30f43335df4 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:47.017585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed0afe-c03a-4033-8a99-ab21e184be2a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:47.019042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 980/1957] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:46 2022] PUT /identity/v3/projects/47f5acbb2176422c91f24759ee8ee931/users/3911e03be7dd449c8b2a45273ec0db39/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:47.020024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83551c4-20f8-4fc3-b596-a30f43335df4 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:47.071061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:47.111251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03d211-4acd-4dea-9bae-5f7c36077735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:47.111251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03d211-4acd-4dea-9bae-5f7c36077735 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:47.111251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03d211-4acd-4dea-9bae-5f7c36077735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:47.111251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03d211-4acd-4dea-9bae-5f7c36077735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:47.139673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:47.160157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83551c4-20f8-4fc3-b596-a30f43335df4 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:47.176279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127bdde-d984-4323-8e65-a62c656d8683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:47.178604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127bdde-d984-4323-8e65-a62c656d8683 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:47.179270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127bdde-d984-4323-8e65-a62c656d8683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:47.179726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127bdde-d984-4323-8e65-a62c656d8683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:47.180566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127bdde-d984-4323-8e65-a62c656d8683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:47.182598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127bdde-d984-4323-8e65-a62c656d8683 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:47.204854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 981/1958] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:47.204854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6acf4e-c790-4a30-acd4-cd884fc741d4 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:47.205339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6acf4e-c790-4a30-acd4-cd884fc741d4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:47.205894 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6acf4e-c790-4a30-acd4-cd884fc741d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:47.209076 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03d211-4acd-4dea-9bae-5f7c36077735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:47.210603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03d211-4acd-4dea-9bae-5f7c36077735 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:47.228089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 978/1959] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:47.228890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03d211-4acd-4dea-9bae-5f7c36077735 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:47.252567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03d211-4acd-4dea-9bae-5f7c36077735 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:47.257093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-5d6acf4e-c790-4a30-acd4-cd884fc741d4 None None] MFA Rules not processed for user `3911e03be7dd449c8b2a45273ec0db39`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:47.257842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d6e05-4f0d-457e-bc6c-a01bf46ad10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:47.258317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5d6acf4e-c790-4a30-acd4-cd884fc741d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:47.258522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d6e05-4f0d-457e-bc6c-a01bf46ad10b admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:47.259020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d6e05-4f0d-457e-bc6c-a01bf46ad10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:47.259318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d6e05-4f0d-457e-bc6c-a01bf46ad10b admin admin] PATH_INFO: `/v3/projects/e50b6fbedfd74b79b4e4fb0f099e24a0/users/2b1a8cb417184b43bb446655a681736a/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:47.262031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d6e05-4f0d-457e-bc6c-a01bf46ad10b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50b6fbedfd74b79b4e4fb0f099e24a0, user_id=2b1a8cb417184b43bb446655a681736a, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:47.264974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d6e05-4f0d-457e-bc6c-a01bf46ad10b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:47.279777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 979/1960] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:47 2022] PUT /identity/v3/projects/e50b6fbedfd74b79b4e4fb0f099e24a0/users/2b1a8cb417184b43bb446655a681736a/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:47.291542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7fcbd-75fb-4b93-a195-818727336dac None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:47.292156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7fcbd-75fb-4b93-a195-818727336dac None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:47.294018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7fcbd-75fb-4b93-a195-818727336dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:47.359860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-33f7fcbd-75fb-4b93-a195-818727336dac None None] MFA Rules not processed for user `2b1a8cb417184b43bb446655a681736a`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:47.361126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-33f7fcbd-75fb-4b93-a195-818727336dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:47.391000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 982/1961] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:47.408470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6acf4e-c790-4a30-acd4-cd884fc741d4 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:47.425016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6acf4e-c790-4a30-acd4-cd884fc741d4 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:47.477145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 980/1962] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:47.493071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad905122-c0a9-4d53-a7d1-e4b101e65976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:47.493784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad905122-c0a9-4d53-a7d1-e4b101e65976 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:47.494123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad905122-c0a9-4d53-a7d1-e4b101e65976 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:47.494400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad905122-c0a9-4d53-a7d1-e4b101e65976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:47.581413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad905122-c0a9-4d53-a7d1-e4b101e65976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:47.583259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad905122-c0a9-4d53-a7d1-e4b101e65976 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:47.594702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 983/1963] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:47.648103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7fcbd-75fb-4b93-a195-818727336dac None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:47.664696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7fcbd-75fb-4b93-a195-818727336dac None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:47.670905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6c58f-e307-4b04-849d-b382df3ad8dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:47.671744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6c58f-e307-4b04-849d-b382df3ad8dc service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:47.672195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6c58f-e307-4b04-849d-b382df3ad8dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:47.672531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6c58f-e307-4b04-849d-b382df3ad8dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:47.746300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6c58f-e307-4b04-849d-b382df3ad8dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:47.749428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6c58f-e307-4b04-849d-b382df3ad8dc service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:47.754284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:47.765562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 981/1964] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:47.886403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad905122-c0a9-4d53-a7d1-e4b101e65976 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:47.890110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6c58f-e307-4b04-849d-b382df3ad8dc service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:47.898766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6c58f-e307-4b04-849d-b382df3ad8dc service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:47.902507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307743b1-13f5-49f4-9a95-6c348690503c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:47.903203 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307743b1-13f5-49f4-9a95-6c348690503c service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:47.903542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307743b1-13f5-49f4-9a95-6c348690503c service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:47.903972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307743b1-13f5-49f4-9a95-6c348690503c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:47.915317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:47.970285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad905122-c0a9-4d53-a7d1-e4b101e65976 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:47.972687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307743b1-13f5-49f4-9a95-6c348690503c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:47.974351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307743b1-13f5-49f4-9a95-6c348690503c service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:47.975700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:47.976382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:47.977176 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:47.977447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] PATH_INFO: `/v3/users/ae143e8b59db4fc481d64fe359b32bb6` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:47.978758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae143e8b59db4fc481d64fe359b32bb6)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:47.980848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:47.985191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 982/1965] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:48.040008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae143e8b59db4fc481d64fe359b32bb6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.054506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae143e8b59db4fc481d64fe359b32bb6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.065008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae143e8b59db4fc481d64fe359b32bb6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.079114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae143e8b59db4fc481d64fe359b32bb6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.079856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae143e8b59db4fc481d64fe359b32bb6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.086704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 984/1966] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:47 2022] DELETE /identity/v3/users/ae143e8b59db4fc481d64fe359b32bb6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:48.091167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1cab2-272c-494d-bbe8-0431ed861d0c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:48.091524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1cab2-272c-494d-bbe8-0431ed861d0c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:48.091870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1cab2-272c-494d-bbe8-0431ed861d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:48.113570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d3d1cab2-272c-494d-bbe8-0431ed861d0c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:48.124765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d3d1cab2-272c-494d-bbe8-0431ed861d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:48.256951 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 983/1967] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:48.275420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:48.277236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:48.386933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791c0be-435d-44b2-ad25-fc1fb3cdc0ba admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:48.386933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d49f6-9fe8-4527-9a5f-9feaa251cab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:48.386933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d49f6-9fe8-4527-9a5f-9feaa251cab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:48.386933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d49f6-9fe8-4527-9a5f-9feaa251cab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:48.387610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d49f6-9fe8-4527-9a5f-9feaa251cab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:48.394819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d49f6-9fe8-4527-9a5f-9feaa251cab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:48.396645 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d49f6-9fe8-4527-9a5f-9feaa251cab2 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:48.408673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 985/1968] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:48.548544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1cab2-272c-494d-bbe8-0431ed861d0c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:48.550006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d3d1cab2-272c-494d-bbe8-0431ed861d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:48.562706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1cab2-272c-494d-bbe8-0431ed861d0c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:48.571616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:48.572376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:48.572767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:48.573134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] PATH_INFO: `/v3/users/32a2d302049c4552928ce2ca0167d538` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:48.576477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a2d302049c4552928ce2ca0167d538)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:48.578514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:48.644699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a2d302049c4552928ce2ca0167d538'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.652554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a2d302049c4552928ce2ca0167d538'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.667867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a2d302049c4552928ce2ca0167d538'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.668935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a2d302049c4552928ce2ca0167d538'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.669847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a2d302049c4552928ce2ca0167d538'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:48.718110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 984/1969] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:48 2022] DELETE /identity/v3/users/32a2d302049c4552928ce2ca0167d538 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:48.737194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8586a-450c-48b0-9067-7cfdcedbb4f8 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:48.737740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8586a-450c-48b0-9067-7cfdcedbb4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:48.737740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8586a-450c-48b0-9067-7cfdcedbb4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:48.739795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:48.744708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:48.803648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:48.830068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-8db8586a-450c-48b0-9067-7cfdcedbb4f8 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:48.834337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8db8586a-450c-48b0-9067-7cfdcedbb4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:48.921606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:48.941740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a479c-14ea-4267-af28-268e37d1447b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:48.953991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:48.956316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:48.956316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:48.956316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] PATH_INFO: `/v3/projects/ac434880f080463cad7365dda60a1a25` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:48.957209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac434880f080463cad7365dda60a1a25)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:48.962985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 986/1970] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:48.963829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:48.973805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8586a-450c-48b0-9067-7cfdcedbb4f8 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:48.976433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8db8586a-450c-48b0-9067-7cfdcedbb4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:49.072506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8586a-450c-48b0-9067-7cfdcedbb4f8 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:49.081537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac434880f080463cad7365dda60a1a25', 'request_id': 'req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.092314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac434880f080463cad7365dda60a1a25', 'request_id': 'req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.099707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac434880f080463cad7365dda60a1a25', 'request_id': 'req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.112088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] The token cache is being invalidate because project ac434880f080463cad7365dda60a1a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:49.112384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.114267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.115220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 985/1971] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:48 2022] DELETE /identity/v3/projects/ac434880f080463cad7365dda60a1a25 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:49.119048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:49.120592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:49.159268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:49.159975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:49.160301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:49.160939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:49.162703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:49.196551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df080a7-9ebf-49d8-aba9-2addaa8c56ec admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:49.201664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:49.202268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:49.202679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:49.202841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] PATH_INFO: `/v3/users/8d29acb533a64b8794158435943da830` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:49.204271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d29acb533a64b8794158435943da830)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:49.206652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:49.248406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:49.250559 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:49.256618 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:49.262200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 987/1972] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:49.263650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d29acb533a64b8794158435943da830'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.273409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d29acb533a64b8794158435943da830'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.294616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d29acb533a64b8794158435943da830'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.295525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d29acb533a64b8794158435943da830'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.296163 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d29acb533a64b8794158435943da830'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:49.320769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 986/1973] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:49 2022] DELETE /identity/v3/users/8d29acb533a64b8794158435943da830 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:50.092311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:50.093907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:50.126320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c14d9c-1303-4e89-9688-021a64a92974 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:50.162784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:50.164405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:50.273974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:50.308218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:50.309064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.309340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.309606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] PATH_INFO: `/v3/projects/72bffafa3da048ef84af4afdb61d9b5c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.310823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=72bffafa3da048ef84af4afdb61d9b5c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:50.313116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:50.317932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83121a26-7726-4d10-823d-ff0c73da0947 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:50.325857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:50.326697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.327165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.328487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] PATH_INFO: `/v3/projects/e9041a2d35794bb3b3e920d9df519ceb` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.330641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9041a2d35794bb3b3e920d9df519ceb)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:50.333069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:50.381673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72bffafa3da048ef84af4afdb61d9b5c', 'request_id': 'req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.390561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72bffafa3da048ef84af4afdb61d9b5c', 'request_id': 'req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.408128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72bffafa3da048ef84af4afdb61d9b5c', 'request_id': 'req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.418946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9041a2d35794bb3b3e920d9df519ceb', 'request_id': 'req-c04c7102-b885-454a-8b63-d8421ea4e55b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.425402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] The token cache is being invalidate because project 72bffafa3da048ef84af4afdb61d9b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:50.425698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.426368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.428123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 988/1974] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:50 2022] DELETE /identity/v3/projects/72bffafa3da048ef84af4afdb61d9b5c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:50.430699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9041a2d35794bb3b3e920d9df519ceb', 'request_id': 'req-c04c7102-b885-454a-8b63-d8421ea4e55b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.432174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:50.433738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:50.439248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9041a2d35794bb3b3e920d9df519ceb', 'request_id': 'req-c04c7102-b885-454a-8b63-d8421ea4e55b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.451890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] The token cache is being invalidate because project e9041a2d35794bb3b3e920d9df519ceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:50.452174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.452836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c04c7102-b885-454a-8b63-d8421ea4e55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.454332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 987/1975] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:50 2022] DELETE /identity/v3/projects/e9041a2d35794bb3b3e920d9df519ceb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:50.459131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d938b3d-c6ed-43e1-bde7-75cf8bd85f94 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.459485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d938b3d-c6ed-43e1-bde7-75cf8bd85f94 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.459857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d938b3d-c6ed-43e1-bde7-75cf8bd85f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.479216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4d938b3d-c6ed-43e1-bde7-75cf8bd85f94 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:50.480448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4d938b3d-c6ed-43e1-bde7-75cf8bd85f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:50.505701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b6ad-ff4f-47cb-bb22-36047c4bb5ce admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:50.539116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:50.565623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 988/1976] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:50.572878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe333d-40ad-4b0f-8ea0-8d4cabd9065b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.573266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe333d-40ad-4b0f-8ea0-8d4cabd9065b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.573648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe333d-40ad-4b0f-8ea0-8d4cabd9065b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.580271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:50.580906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.581186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.581455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] PATH_INFO: `/v3/users/3911e03be7dd449c8b2a45273ec0db39` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.583709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3911e03be7dd449c8b2a45273ec0db39)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:50.589941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:50.595745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0cbe333d-40ad-4b0f-8ea0-8d4cabd9065b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:50.597036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0cbe333d-40ad-4b0f-8ea0-8d4cabd9065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:50.607058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 989/1977] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:50.611782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe333d-40ad-4b0f-8ea0-8d4cabd9065b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:50.623343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe333d-40ad-4b0f-8ea0-8d4cabd9065b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:50.624223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:50.628899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94ae24-531e-4819-8651-73f53dd35190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:50.629592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94ae24-531e-4819-8651-73f53dd35190 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.629945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94ae24-531e-4819-8651-73f53dd35190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.630284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94ae24-531e-4819-8651-73f53dd35190 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.631214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94ae24-531e-4819-8651-73f53dd35190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:50.631363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94ae24-531e-4819-8651-73f53dd35190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:50.632980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94ae24-531e-4819-8651-73f53dd35190 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:50.645432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3911e03be7dd449c8b2a45273ec0db39'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.646138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 990/1978] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:03:50.652110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94ae24-531e-4819-8651-73f53dd35190 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:50.653054 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3911e03be7dd449c8b2a45273ec0db39'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.660468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3911e03be7dd449c8b2a45273ec0db39'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.662499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94ae24-531e-4819-8651-73f53dd35190 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:50.667109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e0578-d142-48ab-aeb6-0c090a43bf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:50.667700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e0578-d142-48ab-aeb6-0c090a43bf0e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.667980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e0578-d142-48ab-aeb6-0c090a43bf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.668251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e0578-d142-48ab-aeb6-0c090a43bf0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.668889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e0578-d142-48ab-aeb6-0c090a43bf0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:50.670788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e0578-d142-48ab-aeb6-0c090a43bf0e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:50.672455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3911e03be7dd449c8b2a45273ec0db39'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.673191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3911e03be7dd449c8b2a45273ec0db39'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:50.679569 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 989/1979] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:50 2022] DELETE /identity/v3/users/3911e03be7dd449c8b2a45273ec0db39 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:50.698591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 991/1980] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:50 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:50.703405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:50.704845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:50.770750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26056e77-f252-4e6b-b98a-b45c43c4691e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:50.852595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adeca8-3d5a-4004-b227-dcc2bf980054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:50.853622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adeca8-3d5a-4004-b227-dcc2bf980054 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.854163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adeca8-3d5a-4004-b227-dcc2bf980054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.854969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adeca8-3d5a-4004-b227-dcc2bf980054 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.855701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adeca8-3d5a-4004-b227-dcc2bf980054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:50.858633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adeca8-3d5a-4004-b227-dcc2bf980054 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:50.887081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 990/1981] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:50 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:50.888759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e0578-d142-48ab-aeb6-0c090a43bf0e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:50.898946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e0578-d142-48ab-aeb6-0c090a43bf0e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:50.903649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1c1bf-7dc0-4813-a20c-10e603668d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:50.904247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1c1bf-7dc0-4813-a20c-10e603668d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.904523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1c1bf-7dc0-4813-a20c-10e603668d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.904798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1c1bf-7dc0-4813-a20c-10e603668d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.905435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1c1bf-7dc0-4813-a20c-10e603668d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:50.907072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1c1bf-7dc0-4813-a20c-10e603668d4c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:50.917461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 992/1982] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:03:50.922110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adeca8-3d5a-4004-b227-dcc2bf980054 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:50.932077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adeca8-3d5a-4004-b227-dcc2bf980054 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:50.937808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e45d34-9d6e-435b-b19a-037bed3363dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:50.938282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e45d34-9d6e-435b-b19a-037bed3363dc admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.938706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e45d34-9d6e-435b-b19a-037bed3363dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.939010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e45d34-9d6e-435b-b19a-037bed3363dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.939649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e45d34-9d6e-435b-b19a-037bed3363dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:50.941274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e45d34-9d6e-435b-b19a-037bed3363dc admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:50.951891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 991/1983] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:50.955955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1c1bf-7dc0-4813-a20c-10e603668d4c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:50.966022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1c1bf-7dc0-4813-a20c-10e603668d4c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:50.970750 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0abee-57a6-47c4-8ffd-c214453fbefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:50.971839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0abee-57a6-47c4-8ffd-c214453fbefc admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:50.971839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0abee-57a6-47c4-8ffd-c214453fbefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:50.972053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0abee-57a6-47c4-8ffd-c214453fbefc admin admin] PATH_INFO: `/v3/projects/d8dfc0820cb94aa99ea8be4e1f747911/users/72cf9b01a2324c7c9df03f5decc34529/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:50.988069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0abee-57a6-47c4-8ffd-c214453fbefc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dfc0820cb94aa99ea8be4e1f747911, user_id=72cf9b01a2324c7c9df03f5decc34529, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:50.990726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0abee-57a6-47c4-8ffd-c214453fbefc admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.011044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 993/1984] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:50 2022] PUT /identity/v3/projects/d8dfc0820cb94aa99ea8be4e1f747911/users/72cf9b01a2324c7c9df03f5decc34529/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:51.015260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6859-08b2-4852-8e88-a083503ab5c1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.015702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6859-08b2-4852-8e88-a083503ab5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.015837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6859-08b2-4852-8e88-a083503ab5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.047190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0abee-57a6-47c4-8ffd-c214453fbefc admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.050443 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-3a6d6859-08b2-4852-8e88-a083503ab5c1 None None] MFA Rules not processed for user `72cf9b01a2324c7c9df03f5decc34529`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:51.051725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3a6d6859-08b2-4852-8e88-a083503ab5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.052802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-63b0abee-57a6-47c4-8ffd-c214453fbefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.074471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:51.127360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 992/1985] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:51.131907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6859-08b2-4852-8e88-a083503ab5c1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.151450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0abee-57a6-47c4-8ffd-c214453fbefc admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.159000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.159845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.160292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.160567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] PATH_INFO: `/v3/projects/47f5acbb2176422c91f24759ee8ee931` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.161714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f5acbb2176422c91f24759ee8ee931)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.163795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.211817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6859-08b2-4852-8e88-a083503ab5c1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.217754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f5acbb2176422c91f24759ee8ee931', 'request_id': 'req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.237610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f5acbb2176422c91f24759ee8ee931', 'request_id': 'req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.245495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f5acbb2176422c91f24759ee8ee931', 'request_id': 'req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.265585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] The token cache is being invalidate because project 47f5acbb2176422c91f24759ee8ee931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:51.265734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.266409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed5fcc9a-d0ba-4a6f-bd36-5eaa25536f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.271004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 994/1986] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:51 2022] DELETE /identity/v3/projects/47f5acbb2176422c91f24759ee8ee931 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:51.278235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0d68d-a7c6-4b29-8b0a-6a0965e9cef9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.278963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0d68d-a7c6-4b29-8b0a-6a0965e9cef9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.280151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0d68d-a7c6-4b29-8b0a-6a0965e9cef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.305210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:51.306667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d3435-9349-41ed-b01e-4c3647e3ba56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.308065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d3435-9349-41ed-b01e-4c3647e3ba56 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.308345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d3435-9349-41ed-b01e-4c3647e3ba56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.309080 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d3435-9349-41ed-b01e-4c3647e3ba56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.309242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d3435-9349-41ed-b01e-4c3647e3ba56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.314858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d3435-9349-41ed-b01e-4c3647e3ba56 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.330293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f6d0d68d-a7c6-4b29-8b0a-6a0965e9cef9 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:51.333993 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f6d0d68d-a7c6-4b29-8b0a-6a0965e9cef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.358988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 995/1987] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:51.361571 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7752944-3953-4921-81d7-07c3f491d0da None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.362029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7752944-3953-4921-81d7-07c3f491d0da None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.363521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7752944-3953-4921-81d7-07c3f491d0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.367539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 993/1988] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:51 2022] POST /identity/v3/projects => generated 387 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:51.371327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d3435-9349-41ed-b01e-4c3647e3ba56 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.374401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b02d3435-9349-41ed-b01e-4c3647e3ba56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.442078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b7752944-3953-4921-81d7-07c3f491d0da None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:51.443339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b7752944-3953-4921-81d7-07c3f491d0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.450475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:51.468296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d3435-9349-41ed-b01e-4c3647e3ba56 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.474180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a0a53-9da4-43fb-9581-3d8ed7095b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.474902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a0a53-9da4-43fb-9581-3d8ed7095b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.475316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a0a53-9da4-43fb-9581-3d8ed7095b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.475696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a0a53-9da4-43fb-9581-3d8ed7095b80 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.476506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a0a53-9da4-43fb-9581-3d8ed7095b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.479335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a0a53-9da4-43fb-9581-3d8ed7095b80 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.505216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 994/1989] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:51 2022] POST /identity/v3/users => generated 300 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:51.506186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a0a53-9da4-43fb-9581-3d8ed7095b80 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.507958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8c6a0a53-9da4-43fb-9581-3d8ed7095b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.523476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a0a53-9da4-43fb-9581-3d8ed7095b80 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.529844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.530612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.531037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.531467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] PATH_INFO: `/v3/users/47fa663054ef4772a379e885844d5e30` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.533125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=47fa663054ef4772a379e885844d5e30)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.535514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.551463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 996/1990] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:51.551887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7752944-3953-4921-81d7-07c3f491d0da None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.566524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7752944-3953-4921-81d7-07c3f491d0da None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.572144 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee0d18-ca23-4f4f-9e82-e3381b271a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.572770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee0d18-ca23-4f4f-9e82-e3381b271a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.573194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee0d18-ca23-4f4f-9e82-e3381b271a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.573620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee0d18-ca23-4f4f-9e82-e3381b271a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.574273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee0d18-ca23-4f4f-9e82-e3381b271a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.576412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee0d18-ca23-4f4f-9e82-e3381b271a26 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.591842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 997/1991] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:51.592715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee0d18-ca23-4f4f-9e82-e3381b271a26 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.608148 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee0d18-ca23-4f4f-9e82-e3381b271a26 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.613043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b90230-16dc-45d9-8a8a-3bb88eeddb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.613655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b90230-16dc-45d9-8a8a-3bb88eeddb3c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.614092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b90230-16dc-45d9-8a8a-3bb88eeddb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.614473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b90230-16dc-45d9-8a8a-3bb88eeddb3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.615213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b90230-16dc-45d9-8a8a-3bb88eeddb3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:51.615520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b90230-16dc-45d9-8a8a-3bb88eeddb3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.617231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b90230-16dc-45d9-8a8a-3bb88eeddb3c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.618533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47fa663054ef4772a379e885844d5e30'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.633023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47fa663054ef4772a379e885844d5e30'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.646262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47fa663054ef4772a379e885844d5e30'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.647814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 998/1992] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:51.648922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b90230-16dc-45d9-8a8a-3bb88eeddb3c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.664339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b90230-16dc-45d9-8a8a-3bb88eeddb3c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.666142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47fa663054ef4772a379e885844d5e30'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.666931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47fa663054ef4772a379e885844d5e30'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:51.669901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f546b-9126-4162-95f1-f00bfe22d2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.670548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f546b-9126-4162-95f1-f00bfe22d2b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.670824 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f546b-9126-4162-95f1-f00bfe22d2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.671128 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f546b-9126-4162-95f1-f00bfe22d2b2 admin admin] PATH_INFO: `/v3/projects/2f4361c72c6b442fb55b0058f094e982/users/f7bf328c96554cea89c4256d923936f7/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.677477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 995/1993] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:51 2022] DELETE /identity/v3/users/47fa663054ef4772a379e885844d5e30 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:51.677477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.677477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.691091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f546b-9126-4162-95f1-f00bfe22d2b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f4361c72c6b442fb55b0058f094e982, user_id=f7bf328c96554cea89c4256d923936f7, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.693971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f546b-9126-4162-95f1-f00bfe22d2b2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.712078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 999/1994] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:51 2022] PUT /identity/v3/projects/2f4361c72c6b442fb55b0058f094e982/users/f7bf328c96554cea89c4256d923936f7/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:51.713876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519a02e-67b1-4fee-aa87-6810d9b86a8f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.714396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519a02e-67b1-4fee-aa87-6810d9b86a8f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.714778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519a02e-67b1-4fee-aa87-6810d9b86a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.735425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b519a02e-67b1-4fee-aa87-6810d9b86a8f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:51.736917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b519a02e-67b1-4fee-aa87-6810d9b86a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.772876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45999e4-3be6-4d2a-8678-9632f83392ce admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.812739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1000/1995] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:51.812739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519a02e-67b1-4fee-aa87-6810d9b86a8f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.813983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b519a02e-67b1-4fee-aa87-6810d9b86a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.847138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519a02e-67b1-4fee-aa87-6810d9b86a8f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.852243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150c2a9-513e-4ae9-ae86-4942e3bce765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.853122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150c2a9-513e-4ae9-ae86-4942e3bce765 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.853122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150c2a9-513e-4ae9-ae86-4942e3bce765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.853384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150c2a9-513e-4ae9-ae86-4942e3bce765 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.854017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150c2a9-513e-4ae9-ae86-4942e3bce765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.855647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150c2a9-513e-4ae9-ae86-4942e3bce765 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.861987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb05614-6c6a-425c-b798-b2ef9fe62cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.863237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb05614-6c6a-425c-b798-b2ef9fe62cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.863950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb05614-6c6a-425c-b798-b2ef9fe62cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.863950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb05614-6c6a-425c-b798-b2ef9fe62cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.864820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb05614-6c6a-425c-b798-b2ef9fe62cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.867284 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb05614-6c6a-425c-b798-b2ef9fe62cf7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.867422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1001/1996] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:51.868248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150c2a9-513e-4ae9-ae86-4942e3bce765 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.869936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6150c2a9-513e-4ae9-ae86-4942e3bce765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:51.874476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:51.899675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 996/1997] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:51 2022] POST /identity/v3/projects => generated 359 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:51.900404 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb05614-6c6a-425c-b798-b2ef9fe62cf7 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.917795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:51.934129 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150c2a9-513e-4ae9-ae86-4942e3bce765 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.961463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb05614-6c6a-425c-b798-b2ef9fe62cf7 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:51.966909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20beed-1db8-4d9c-a423-ba9e8b98b80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.967596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20beed-1db8-4d9c-a423-ba9e8b98b80a admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.967966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20beed-1db8-4d9c-a423-ba9e8b98b80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.968246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20beed-1db8-4d9c-a423-ba9e8b98b80a admin admin] PATH_INFO: `/v3/projects/2f4361c72c6b442fb55b0058f094e982/users/f7bf328c96554cea89c4256d923936f7/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.970218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20beed-1db8-4d9c-a423-ba9e8b98b80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f4361c72c6b442fb55b0058f094e982, user_id=f7bf328c96554cea89c4256d923936f7, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.972943 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20beed-1db8-4d9c-a423-ba9e8b98b80a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:51.983122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 997/1998] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:51 2022] PUT /identity/v3/projects/2f4361c72c6b442fb55b0058f094e982/users/f7bf328c96554cea89c4256d923936f7/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:51.983896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20beed-1db8-4d9c-a423-ba9e8b98b80a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:51.984408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13318721-2cfd-4e37-84f1-eeb4d0a1f244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:51.985050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13318721-2cfd-4e37-84f1-eeb4d0a1f244 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:51.985479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13318721-2cfd-4e37-84f1-eeb4d0a1f244 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:51.985841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13318721-2cfd-4e37-84f1-eeb4d0a1f244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:51.996157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13318721-2cfd-4e37-84f1-eeb4d0a1f244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:51.998125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13318721-2cfd-4e37-84f1-eeb4d0a1f244 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.008439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1002/1999] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:52.009620 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfdb17-d141-4491-805c-99f695c8e477 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.009896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfdb17-d141-4491-805c-99f695c8e477 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.010164 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfdb17-d141-4491-805c-99f695c8e477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.039999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e7bfdb17-d141-4491-805c-99f695c8e477 None None] MFA Rules not processed for user `f7bf328c96554cea89c4256d923936f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:52.041144 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e7bfdb17-d141-4491-805c-99f695c8e477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.052853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20beed-1db8-4d9c-a423-ba9e8b98b80a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.058048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8876aa-20ba-4e30-b183-9b56fa35c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.058653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8876aa-20ba-4e30-b183-9b56fa35c42b admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.058928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8876aa-20ba-4e30-b183-9b56fa35c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.060504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8876aa-20ba-4e30-b183-9b56fa35c42b admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.060504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8876aa-20ba-4e30-b183-9b56fa35c42b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.062449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8876aa-20ba-4e30-b183-9b56fa35c42b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.084867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 998/2000] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:51 2022] POST /identity/v3/users => generated 280 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:52.089265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8876aa-20ba-4e30-b183-9b56fa35c42b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.099797 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8876aa-20ba-4e30-b183-9b56fa35c42b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.104956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718690b-2ba6-4f7d-944b-cb53deae2eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.105631 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718690b-2ba6-4f7d-944b-cb53deae2eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.105979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718690b-2ba6-4f7d-944b-cb53deae2eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.106322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718690b-2ba6-4f7d-944b-cb53deae2eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.107210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718690b-2ba6-4f7d-944b-cb53deae2eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.108776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718690b-2ba6-4f7d-944b-cb53deae2eb4 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.126920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1003/2001] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:52.127769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 999/2002] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:52.128192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfdb17-d141-4491-805c-99f695c8e477 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.135640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718690b-2ba6-4f7d-944b-cb53deae2eb4 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.136465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfdb17-d141-4491-805c-99f695c8e477 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.138293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e718690b-2ba6-4f7d-944b-cb53deae2eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.141338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4aac9-f561-4e69-aff1-f0e9969a0ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.141950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4aac9-f561-4e69-aff1-f0e9969a0ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.142230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4aac9-f561-4e69-aff1-f0e9969a0ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.142501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4aac9-f561-4e69-aff1-f0e9969a0ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.143172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4aac9-f561-4e69-aff1-f0e9969a0ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.144751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4aac9-f561-4e69-aff1-f0e9969a0ca6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.157631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1004/2003] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:03:52.160739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4aac9-f561-4e69-aff1-f0e9969a0ca6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.176522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4aac9-f561-4e69-aff1-f0e9969a0ca6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.187499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b5e0a-6993-4ba6-9b30-f2f178c9ae2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.188666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b5e0a-6993-4ba6-9b30-f2f178c9ae2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.189401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b5e0a-6993-4ba6-9b30-f2f178c9ae2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.189827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b5e0a-6993-4ba6-9b30-f2f178c9ae2a admin admin] PATH_INFO: `/v3/projects/1a94c5cc0a64419fa231abdac6ba4668/users/6aec39d36a4e4c15b945c1f70ef0e590/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.203762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718690b-2ba6-4f7d-944b-cb53deae2eb4 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.208644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b5e0a-6993-4ba6-9b30-f2f178c9ae2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a94c5cc0a64419fa231abdac6ba4668, user_id=6aec39d36a4e4c15b945c1f70ef0e590, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.209884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ab9cd-a4fc-420d-ad64-169af54f909a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.211341 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ab9cd-a4fc-420d-ad64-169af54f909a service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.211682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b5e0a-6993-4ba6-9b30-f2f178c9ae2a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.211828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ab9cd-a4fc-420d-ad64-169af54f909a service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.212109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ab9cd-a4fc-420d-ad64-169af54f909a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.223151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ab9cd-a4fc-420d-ad64-169af54f909a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.224902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ab9cd-a4fc-420d-ad64-169af54f909a service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.228179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1005/2004] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:52 2022] PUT /identity/v3/projects/1a94c5cc0a64419fa231abdac6ba4668/users/6aec39d36a4e4c15b945c1f70ef0e590/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:52.237752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1000/2005] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:52.239517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813a8a2-6cab-41ec-8cb8-b67e52c0e5d8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.239807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813a8a2-6cab-41ec-8cb8-b67e52c0e5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.240082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813a8a2-6cab-41ec-8cb8-b67e52c0e5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.269124 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0813a8a2-6cab-41ec-8cb8-b67e52c0e5d8 None None] MFA Rules not processed for user `6aec39d36a4e4c15b945c1f70ef0e590`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:52.271045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0813a8a2-6cab-41ec-8cb8-b67e52c0e5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.321638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ab9cd-a4fc-420d-ad64-169af54f909a service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.322175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:52.329496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:52.337440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ab9cd-a4fc-420d-ad64-169af54f909a service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.363371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1006/2006] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:52.364153 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813a8a2-6cab-41ec-8cb8-b67e52c0e5d8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.365461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0813a8a2-6cab-41ec-8cb8-b67e52c0e5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.426469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.427138 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.427419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.427692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] PATH_INFO: `/v3/users/72cf9b01a2324c7c9df03f5decc34529` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.428842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72cf9b01a2324c7c9df03f5decc34529)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.430981 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.460691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813a8a2-6cab-41ec-8cb8-b67e52c0e5d8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.466001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.466721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.467102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.467533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] PATH_INFO: `/v3/projects/01fdc9650a53413f9e9a15f96cabf43b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.468765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01fdc9650a53413f9e9a15f96cabf43b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.470738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.488355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72cf9b01a2324c7c9df03f5decc34529'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.505972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72cf9b01a2324c7c9df03f5decc34529'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.514669 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72cf9b01a2324c7c9df03f5decc34529'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.532584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72cf9b01a2324c7c9df03f5decc34529'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.533380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72cf9b01a2324c7c9df03f5decc34529'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.543083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01fdc9650a53413f9e9a15f96cabf43b', 'request_id': 'req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.544696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1001/2007] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:52 2022] DELETE /identity/v3/users/72cf9b01a2324c7c9df03f5decc34529 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:52.545860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.547373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.553600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01fdc9650a53413f9e9a15f96cabf43b', 'request_id': 'req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.561392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01fdc9650a53413f9e9a15f96cabf43b', 'request_id': 'req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.573171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] The token cache is being invalidate because project 01fdc9650a53413f9e9a15f96cabf43b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:52.573463 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.574348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-88bf8ac2-19ec-4d79-b6be-b53b31ba7637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.576644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1007/2008] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:52 2022] DELETE /identity/v3/projects/01fdc9650a53413f9e9a15f96cabf43b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:52.580139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7082d-2e0b-439d-92a5-802e2cd8803f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.580139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7082d-2e0b-439d-92a5-802e2cd8803f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.580139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7082d-2e0b-439d-92a5-802e2cd8803f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.597748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8bf7082d-2e0b-439d-92a5-802e2cd8803f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:52.598926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8bf7082d-2e0b-439d-92a5-802e2cd8803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.621293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffd2ac-f736-4953-8c22-a7801764d342 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.625701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d217696-96c1-4986-8b46-ed3047b9fbe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.626298 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d217696-96c1-4986-8b46-ed3047b9fbe4 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.626572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d217696-96c1-4986-8b46-ed3047b9fbe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.626842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d217696-96c1-4986-8b46-ed3047b9fbe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.628564 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7d217696-96c1-4986-8b46-ed3047b9fbe4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.664599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1008/2009] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:52.666075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223a7bb-7c28-4ee9-9329-bf50c9b9155d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.666339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223a7bb-7c28-4ee9-9329-bf50c9b9155d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.666760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223a7bb-7c28-4ee9-9329-bf50c9b9155d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.675197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:52.685676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:52.691454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6223a7bb-7c28-4ee9-9329-bf50c9b9155d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:52.693604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6223a7bb-7c28-4ee9-9329-bf50c9b9155d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.703359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1009/2010] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:52.704507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223a7bb-7c28-4ee9-9329-bf50c9b9155d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.708493 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6223a7bb-7c28-4ee9-9329-bf50c9b9155d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.711984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d217696-96c1-4986-8b46-ed3047b9fbe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.713546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d217696-96c1-4986-8b46-ed3047b9fbe4 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.724727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1002/2011] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:52.726283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777103-fdd8-4b05-90c3-9e28e9af9c99 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.726712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777103-fdd8-4b05-90c3-9e28e9af9c99 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.727162 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777103-fdd8-4b05-90c3-9e28e9af9c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.756560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-b7777103-fdd8-4b05-90c3-9e28e9af9c99 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:52.757936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b7777103-fdd8-4b05-90c3-9e28e9af9c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.766527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1003/2012] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:52.767572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777103-fdd8-4b05-90c3-9e28e9af9c99 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.768933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b7777103-fdd8-4b05-90c3-9e28e9af9c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.775605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223a7bb-7c28-4ee9-9329-bf50c9b9155d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.780877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f08d2e-62d0-4b6a-a1df-0ae8d79825d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.781398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777103-fdd8-4b05-90c3-9e28e9af9c99 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.781546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f08d2e-62d0-4b6a-a1df-0ae8d79825d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.781772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f08d2e-62d0-4b6a-a1df-0ae8d79825d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.782046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f08d2e-62d0-4b6a-a1df-0ae8d79825d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.786377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.787107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.787453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.787794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] PATH_INFO: `/v3/users/6aec39d36a4e4c15b945c1f70ef0e590` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.789031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aec39d36a4e4c15b945c1f70ef0e590)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.791049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.792048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f08d2e-62d0-4b6a-a1df-0ae8d79825d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.793489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f08d2e-62d0-4b6a-a1df-0ae8d79825d6 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.810525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1010/2013] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:52.811481 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f08d2e-62d0-4b6a-a1df-0ae8d79825d6 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.821350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f08d2e-62d0-4b6a-a1df-0ae8d79825d6 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.829264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e5630-a8f1-4939-82cd-2d5e52980795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.829868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e5630-a8f1-4939-82cd-2d5e52980795 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.830444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e5630-a8f1-4939-82cd-2d5e52980795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.830573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e5630-a8f1-4939-82cd-2d5e52980795 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.831319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e5630-a8f1-4939-82cd-2d5e52980795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:52.831509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e5630-a8f1-4939-82cd-2d5e52980795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.833305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e5630-a8f1-4939-82cd-2d5e52980795 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.847678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aec39d36a4e4c15b945c1f70ef0e590'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.854360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1011/2014] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:52.855060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e5630-a8f1-4939-82cd-2d5e52980795 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.870066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e5630-a8f1-4939-82cd-2d5e52980795 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:52.870838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aec39d36a4e4c15b945c1f70ef0e590'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.876687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bb2b6-048a-4a93-a674-a6a783d0a17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:52.877371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bb2b6-048a-4a93-a674-a6a783d0a17b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.877652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bb2b6-048a-4a93-a674-a6a783d0a17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.878417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bb2b6-048a-4a93-a674-a6a783d0a17b admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.879276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bb2b6-048a-4a93-a674-a6a783d0a17b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:52.881047 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bb2b6-048a-4a93-a674-a6a783d0a17b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:52.882438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aec39d36a4e4c15b945c1f70ef0e590'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.909479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aec39d36a4e4c15b945c1f70ef0e590'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.910312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-38ef4862-d8ff-44a8-a221-89634ea0cbb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aec39d36a4e4c15b945c1f70ef0e590'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:52.912812 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1012/2015] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:52 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:52.916934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bb2b6-048a-4a93-a674-a6a783d0a17b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:52.917331 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1004/2016] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:52 2022] DELETE /identity/v3/users/6aec39d36a4e4c15b945c1f70ef0e590 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:52.920995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-069bb2b6-048a-4a93-a674-a6a783d0a17b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:52.921863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7343-989a-4a78-b420-a65cecab2036 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:52.921863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7343-989a-4a78-b420-a65cecab2036 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:52.922157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7343-989a-4a78-b420-a65cecab2036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:52.943276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-87ce7343-989a-4a78-b420-a65cecab2036 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:52.944538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-87ce7343-989a-4a78-b420-a65cecab2036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:53.014090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1005/2017] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:53.016326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bb2b6-048a-4a93-a674-a6a783d0a17b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:53.020619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7343-989a-4a78-b420-a65cecab2036 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:53.022094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-87ce7343-989a-4a78-b420-a65cecab2036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:53.023312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152578b-49c6-4fa6-880a-5ecf1e372f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:53.023788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152578b-49c6-4fa6-880a-5ecf1e372f12 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.024066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152578b-49c6-4fa6-880a-5ecf1e372f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.024333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152578b-49c6-4fa6-880a-5ecf1e372f12 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.025085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152578b-49c6-4fa6-880a-5ecf1e372f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:53.027250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152578b-49c6-4fa6-880a-5ecf1e372f12 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:53.054443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1013/2018] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:52 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:53.058447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152578b-49c6-4fa6-880a-5ecf1e372f12 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:53.070753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152578b-49c6-4fa6-880a-5ecf1e372f12 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:53.071908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:53.077803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67298144-8a83-40a8-bc7a-53458d902160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:53.078558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67298144-8a83-40a8-bc7a-53458d902160 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.081476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67298144-8a83-40a8-bc7a-53458d902160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.082145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67298144-8a83-40a8-bc7a-53458d902160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.083397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67298144-8a83-40a8-bc7a-53458d902160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:53.085195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67298144-8a83-40a8-bc7a-53458d902160 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:53.085529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7343-989a-4a78-b420-a65cecab2036 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:53.089959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283e571-536e-43fa-aad7-f37f9fcd13b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:53.090666 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283e571-536e-43fa-aad7-f37f9fcd13b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.091063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283e571-536e-43fa-aad7-f37f9fcd13b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.091406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283e571-536e-43fa-aad7-f37f9fcd13b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.096843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1014/2019] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:03:53.100884 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67298144-8a83-40a8-bc7a-53458d902160 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:53.104418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283e571-536e-43fa-aad7-f37f9fcd13b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:53.105885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283e571-536e-43fa-aad7-f37f9fcd13b7 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:53.111851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67298144-8a83-40a8-bc7a-53458d902160 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:53.117104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc6c61-8f05-4b91-acb1-a6c19eb3d6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:53.117830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc6c61-8f05-4b91-acb1-a6c19eb3d6da admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.117947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1006/2020] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:53.118239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc6c61-8f05-4b91-acb1-a6c19eb3d6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.118516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc6c61-8f05-4b91-acb1-a6c19eb3d6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.119255 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc6c61-8f05-4b91-acb1-a6c19eb3d6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:53.120857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc6c61-8f05-4b91-acb1-a6c19eb3d6da admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:53.139215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1015/2021] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:03:53.140530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283e571-536e-43fa-aad7-f37f9fcd13b7 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:53.151158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283e571-536e-43fa-aad7-f37f9fcd13b7 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:53.151768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc6c61-8f05-4b91-acb1-a6c19eb3d6da admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:53.153588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-eddc6c61-8f05-4b91-acb1-a6c19eb3d6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:53.156173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96231d-0826-4d03-96ce-edac86a15b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:53.156835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96231d-0826-4d03-96ce-edac86a15b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.157110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96231d-0826-4d03-96ce-edac86a15b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.157384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96231d-0826-4d03-96ce-edac86a15b21 admin admin] PATH_INFO: `/v3/projects/cf1721023e1545028ac04c9620fecaae/users/6340a2593f6b4901bc8c4b954e32476e/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.178427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc6c61-8f05-4b91-acb1-a6c19eb3d6da admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:53.183657 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:53.184464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.185336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.185635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] PATH_INFO: `/v3/projects/d8dfc0820cb94aa99ea8be4e1f747911` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.186463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96231d-0826-4d03-96ce-edac86a15b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf1721023e1545028ac04c9620fecaae, user_id=6340a2593f6b4901bc8c4b954e32476e, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:53.187169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8dfc0820cb94aa99ea8be4e1f747911)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:53.189206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:53.189309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96231d-0826-4d03-96ce-edac86a15b21 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:53.217306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1007/2022] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:53 2022] PUT /identity/v3/projects/cf1721023e1545028ac04c9620fecaae/users/6340a2593f6b4901bc8c4b954e32476e/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:53.219093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e3fd7-2cd2-4de9-b363-3f99bf32210b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.219565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e3fd7-2cd2-4de9-b363-3f99bf32210b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.220025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e3fd7-2cd2-4de9-b363-3f99bf32210b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.231044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:53.248161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8dfc0820cb94aa99ea8be4e1f747911', 'request_id': 'req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.256509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8dfc0820cb94aa99ea8be4e1f747911', 'request_id': 'req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.259093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ae7e3fd7-2cd2-4de9-b363-3f99bf32210b None None] MFA Rules not processed for user `6340a2593f6b4901bc8c4b954e32476e`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:53.260497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ae7e3fd7-2cd2-4de9-b363-3f99bf32210b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:53.263599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8dfc0820cb94aa99ea8be4e1f747911', 'request_id': 'req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.274881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] The token cache is being invalidate because project d8dfc0820cb94aa99ea8be4e1f747911 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:53.275289 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.276008 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.277615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1016/2023] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:53 2022] DELETE /identity/v3/projects/d8dfc0820cb94aa99ea8be4e1f747911 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:53.282136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:53.283676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:53.353341 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1008/2024] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:53.357998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802ab4f-4697-411f-9b32-ab5fb94bd4d3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:53.363960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:53.364985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.365450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.365878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] PATH_INFO: `/v3/users/f7bf328c96554cea89c4256d923936f7` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.367711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7bf328c96554cea89c4256d923936f7)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:53.372800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:53.373652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e3fd7-2cd2-4de9-b363-3f99bf32210b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:53.378704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ae7e3fd7-2cd2-4de9-b363-3f99bf32210b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:53.447908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7bf328c96554cea89c4256d923936f7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.448661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e3fd7-2cd2-4de9-b363-3f99bf32210b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:53.455956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7bf328c96554cea89c4256d923936f7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.456328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8add8-53f6-45d6-81bb-99df31475ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:53.456945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8add8-53f6-45d6-81bb-99df31475ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.457227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8add8-53f6-45d6-81bb-99df31475ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.457563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8add8-53f6-45d6-81bb-99df31475ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.467337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8add8-53f6-45d6-81bb-99df31475ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:53.468859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8add8-53f6-45d6-81bb-99df31475ec5 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:53.472230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7bf328c96554cea89c4256d923936f7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.473113 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7bf328c96554cea89c4256d923936f7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.473798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-412e408b-527c-4d0a-9372-ba2409b8ade7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7bf328c96554cea89c4256d923936f7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.479522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1009/2025] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:53.480562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8add8-53f6-45d6-81bb-99df31475ec5 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:53.482846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5cc8add8-53f6-45d6-81bb-99df31475ec5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:53.488801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1017/2026] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:53 2022] DELETE /identity/v3/users/f7bf328c96554cea89c4256d923936f7 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:53.499046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8add8-53f6-45d6-81bb-99df31475ec5 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:53.553988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:53.571276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:53.571934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.572256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.572583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] PATH_INFO: `/v3/projects/1a94c5cc0a64419fa231abdac6ba4668` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.573938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a94c5cc0a64419fa231abdac6ba4668)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:53.576763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:53.661259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a94c5cc0a64419fa231abdac6ba4668', 'request_id': 'req-3abae3e2-50f8-4abb-9f78-bbc0c036860c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.675214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a94c5cc0a64419fa231abdac6ba4668', 'request_id': 'req-3abae3e2-50f8-4abb-9f78-bbc0c036860c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.695912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a94c5cc0a64419fa231abdac6ba4668', 'request_id': 'req-3abae3e2-50f8-4abb-9f78-bbc0c036860c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.713464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] The token cache is being invalidate because project 1a94c5cc0a64419fa231abdac6ba4668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:53.713899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.715409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:53.722627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1010/2027] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:53 2022] DELETE /identity/v3/projects/1a94c5cc0a64419fa231abdac6ba4668 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:53.724528 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536f75-45df-457a-b09d-ccd1ceb12988 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.726231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536f75-45df-457a-b09d-ccd1ceb12988 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.726678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536f75-45df-457a-b09d-ccd1ceb12988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.781176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-34536f75-45df-457a-b09d-ccd1ceb12988 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:53.782552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-34536f75-45df-457a-b09d-ccd1ceb12988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:53.864188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:53.945508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1018/2028] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:53.954550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:53.956574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:53.966891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7f6c5-a9f6-44de-94e0-1d547abdfa12 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:53.971645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7f6c5-a9f6-44de-94e0-1d547abdfa12 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:53.971768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7f6c5-a9f6-44de-94e0-1d547abdfa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:53.983237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abae3e2-50f8-4abb-9f78-bbc0c036860c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.006032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.007938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.023945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.024234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] PATH_INFO: `/v3/projects/2f4361c72c6b442fb55b0058f094e982` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.025659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f4361c72c6b442fb55b0058f094e982)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.027714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.056990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-28c7f6c5-a9f6-44de-94e0-1d547abdfa12 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:54.056990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-28c7f6c5-a9f6-44de-94e0-1d547abdfa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:54.077388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1019/2029] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:54.082095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7f6c5-a9f6-44de-94e0-1d547abdfa12 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.089367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f4361c72c6b442fb55b0058f094e982', 'request_id': 'req-a86c98bf-920d-48d5-a6e6-64c0e069e697'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:54.094969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7f6c5-a9f6-44de-94e0-1d547abdfa12 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.097261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f4361c72c6b442fb55b0058f094e982', 'request_id': 'req-a86c98bf-920d-48d5-a6e6-64c0e069e697'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:54.107745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f4361c72c6b442fb55b0058f094e982', 'request_id': 'req-a86c98bf-920d-48d5-a6e6-64c0e069e697'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:54.142411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] The token cache is being invalidate because project 2f4361c72c6b442fb55b0058f094e982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:54.143504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:54.144199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:54.148967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1011/2030] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:53 2022] DELETE /identity/v3/projects/2f4361c72c6b442fb55b0058f094e982 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:54.148967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.148967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:54.169816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89768f1-ae77-400c-a055-5c8bfe66bc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.170588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89768f1-ae77-400c-a055-5c8bfe66bc1d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.170979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89768f1-ae77-400c-a055-5c8bfe66bc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.171683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89768f1-ae77-400c-a055-5c8bfe66bc1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.172081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89768f1-ae77-400c-a055-5c8bfe66bc1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:54.172388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89768f1-ae77-400c-a055-5c8bfe66bc1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.174125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89768f1-ae77-400c-a055-5c8bfe66bc1d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.186984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1020/2031] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:03:54.187943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb2c3a-81a8-4ec4-a8cc-61153e200e0b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.188276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb2c3a-81a8-4ec4-a8cc-61153e200e0b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.188614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb2c3a-81a8-4ec4-a8cc-61153e200e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.209338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4ceb2c3a-81a8-4ec4-a8cc-61153e200e0b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:54.210536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4ceb2c3a-81a8-4ec4-a8cc-61153e200e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:54.214709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c98bf-920d-48d5-a6e6-64c0e069e697 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.218393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1021/2032] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:54.219090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766f0d1-5cd1-4edf-87df-e47935f3f6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.219223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb2c3a-81a8-4ec4-a8cc-61153e200e0b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.219315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766f0d1-5cd1-4edf-87df-e47935f3f6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.219594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766f0d1-5cd1-4edf-87df-e47935f3f6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.219940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766f0d1-5cd1-4edf-87df-e47935f3f6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.223972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4ceb2c3a-81a8-4ec4-a8cc-61153e200e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:54.224099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6766f0d1-5cd1-4edf-87df-e47935f3f6c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:54.236486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb2c3a-81a8-4ec4-a8cc-61153e200e0b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.243091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f070f-ee37-4c81-b032-c0179c77959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.244018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f070f-ee37-4c81-b032-c0179c77959b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.244292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f070f-ee37-4c81-b032-c0179c77959b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.244661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f070f-ee37-4c81-b032-c0179c77959b admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.244997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:54.245364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f070f-ee37-4c81-b032-c0179c77959b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.248072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f070f-ee37-4c81-b032-c0179c77959b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.278367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1022/2033] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:54 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:54.279873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294c160-98ab-4cb0-849a-95500b6cfb7d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.280250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294c160-98ab-4cb0-849a-95500b6cfb7d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.280590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294c160-98ab-4cb0-849a-95500b6cfb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.295829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766f0d1-5cd1-4edf-87df-e47935f3f6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.297267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766f0d1-5cd1-4edf-87df-e47935f3f6c2 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.300219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4294c160-98ab-4cb0-849a-95500b6cfb7d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:54.301406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4294c160-98ab-4cb0-849a-95500b6cfb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:54.306851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1012/2034] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:54.307655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766f0d1-5cd1-4edf-87df-e47935f3f6c2 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.317893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766f0d1-5cd1-4edf-87df-e47935f3f6c2 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.368369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1023/2035] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:54.372761 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294c160-98ab-4cb0-849a-95500b6cfb7d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.380327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52f2a3-36e9-4301-b069-55ab75b0facd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.380982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52f2a3-36e9-4301-b069-55ab75b0facd admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.381330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52f2a3-36e9-4301-b069-55ab75b0facd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.381671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52f2a3-36e9-4301-b069-55ab75b0facd admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.382378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52f2a3-36e9-4301-b069-55ab75b0facd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.383064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294c160-98ab-4cb0-849a-95500b6cfb7d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.384593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52f2a3-36e9-4301-b069-55ab75b0facd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.388949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a75afc-18a5-4e83-ab7c-ba8e8637b679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.389552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a75afc-18a5-4e83-ab7c-ba8e8637b679 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.389830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a75afc-18a5-4e83-ab7c-ba8e8637b679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.390102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a75afc-18a5-4e83-ab7c-ba8e8637b679 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.390724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a75afc-18a5-4e83-ab7c-ba8e8637b679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:54.391049 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a75afc-18a5-4e83-ab7c-ba8e8637b679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.392606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a75afc-18a5-4e83-ab7c-ba8e8637b679 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.412382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1024/2036] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:03:54.413538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a75afc-18a5-4e83-ab7c-ba8e8637b679 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.422214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1013/2037] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:54 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:54.425100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a75afc-18a5-4e83-ab7c-ba8e8637b679 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.426407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52f2a3-36e9-4301-b069-55ab75b0facd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.430199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa4c69-9f87-4fc2-b02e-92b01b14ce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.431037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa4c69-9f87-4fc2-b02e-92b01b14ce19 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.431166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa4c69-9f87-4fc2-b02e-92b01b14ce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.431367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa4c69-9f87-4fc2-b02e-92b01b14ce19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.431987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa4c69-9f87-4fc2-b02e-92b01b14ce19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.434016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa4c69-9f87-4fc2-b02e-92b01b14ce19 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.436375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52f2a3-36e9-4301-b069-55ab75b0facd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.442106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a828622-949c-4a69-b01c-33c63bbc00c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.443456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a828622-949c-4a69-b01c-33c63bbc00c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.443798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a828622-949c-4a69-b01c-33c63bbc00c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.444079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a828622-949c-4a69-b01c-33c63bbc00c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.444728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a828622-949c-4a69-b01c-33c63bbc00c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.446804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a828622-949c-4a69-b01c-33c63bbc00c0 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.459352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1014/2038] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:03:54.464027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a828622-949c-4a69-b01c-33c63bbc00c0 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.467274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1025/2039] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:54 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:54.471296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa4c69-9f87-4fc2-b02e-92b01b14ce19 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.476455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:54.480165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a828622-949c-4a69-b01c-33c63bbc00c0 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.543134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0788597-5f2e-4b3c-bd13-a864fbe05b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.543956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788597-5f2e-4b3c-bd13-a864fbe05b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.544105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa4c69-9f87-4fc2-b02e-92b01b14ce19 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.544425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788597-5f2e-4b3c-bd13-a864fbe05b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.544793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788597-5f2e-4b3c-bd13-a864fbe05b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.545544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0788597-5f2e-4b3c-bd13-a864fbe05b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.547453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0788597-5f2e-4b3c-bd13-a864fbe05b70 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.549383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513e993-d42a-41a0-99c7-da8c54a200e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.550098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513e993-d42a-41a0-99c7-da8c54a200e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.550372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513e993-d42a-41a0-99c7-da8c54a200e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.550638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513e993-d42a-41a0-99c7-da8c54a200e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.551295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513e993-d42a-41a0-99c7-da8c54a200e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.553567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513e993-d42a-41a0-99c7-da8c54a200e6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.558452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1015/2040] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:54.564016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0788597-5f2e-4b3c-bd13-a864fbe05b70 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.576865 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1026/2041] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:54 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:54.581343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513e993-d42a-41a0-99c7-da8c54a200e6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.582635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0788597-5f2e-4b3c-bd13-a864fbe05b70 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.588193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b86a35-b7df-4b09-9b02-0b51970080cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.588989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b86a35-b7df-4b09-9b02-0b51970080cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.589297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b86a35-b7df-4b09-9b02-0b51970080cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.589637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b86a35-b7df-4b09-9b02-0b51970080cb admin admin] PATH_INFO: `/v3/projects/76f7f3766f4a44c4b3859b8199a670da/users/7ecb55aabbe64be7907123772c359287/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.593237 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513e993-d42a-41a0-99c7-da8c54a200e6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.598192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df860988-4406-49ef-bfaf-c9e1df1369a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.598788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df860988-4406-49ef-bfaf-c9e1df1369a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.599099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df860988-4406-49ef-bfaf-c9e1df1369a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.599367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df860988-4406-49ef-bfaf-c9e1df1369a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.599998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df860988-4406-49ef-bfaf-c9e1df1369a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.601578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df860988-4406-49ef-bfaf-c9e1df1369a9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.606307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b86a35-b7df-4b09-9b02-0b51970080cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f7f3766f4a44c4b3859b8199a670da, user_id=7ecb55aabbe64be7907123772c359287, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.610536 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b86a35-b7df-4b09-9b02-0b51970080cb admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.616159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1027/2042] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:03:54.617872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df860988-4406-49ef-bfaf-c9e1df1369a9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.631557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df860988-4406-49ef-bfaf-c9e1df1369a9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.641989 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1016/2043] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:54 2022] PUT /identity/v3/projects/76f7f3766f4a44c4b3859b8199a670da/users/7ecb55aabbe64be7907123772c359287/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:54.647095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0d69e-f802-46ef-9a24-3337a4a8c388 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.647480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0d69e-f802-46ef-9a24-3337a4a8c388 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.647910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0d69e-f802-46ef-9a24-3337a4a8c388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.661817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:54.669594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-13f0d69e-f802-46ef-9a24-3337a4a8c388 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:54.671164 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-13f0d69e-f802-46ef-9a24-3337a4a8c388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:54.702479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06063875-97f5-4673-9acd-67e73860aefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.703408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06063875-97f5-4673-9acd-67e73860aefb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.703693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06063875-97f5-4673-9acd-67e73860aefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.703970 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06063875-97f5-4673-9acd-67e73860aefb admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.705165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06063875-97f5-4673-9acd-67e73860aefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.711577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06063875-97f5-4673-9acd-67e73860aefb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.727205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1028/2044] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:54.728073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06063875-97f5-4673-9acd-67e73860aefb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.746060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1017/2045] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:54.747918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06063875-97f5-4673-9acd-67e73860aefb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.753271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd108cf3-c952-49f7-8eea-94d076a8a55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.754002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd108cf3-c952-49f7-8eea-94d076a8a55e admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.754370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd108cf3-c952-49f7-8eea-94d076a8a55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.754715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd108cf3-c952-49f7-8eea-94d076a8a55e admin admin] PATH_INFO: `/v3/projects/1231cc02be374a239d73daba2eaf4a75/users/e0389958e0ac4690a3d1630a27d2cfd1/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.756492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0d69e-f802-46ef-9a24-3337a4a8c388 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.759040 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-13f0d69e-f802-46ef-9a24-3337a4a8c388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:54.772423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd108cf3-c952-49f7-8eea-94d076a8a55e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1231cc02be374a239d73daba2eaf4a75, user_id=e0389958e0ac4690a3d1630a27d2cfd1, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.775165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd108cf3-c952-49f7-8eea-94d076a8a55e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.789404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1029/2046] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:54 2022] PUT /identity/v3/projects/1231cc02be374a239d73daba2eaf4a75/users/e0389958e0ac4690a3d1630a27d2cfd1/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:54.800217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bb00c-55ac-434e-bf03-5bc9f17dac97 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.800699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bb00c-55ac-434e-bf03-5bc9f17dac97 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.801040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bb00c-55ac-434e-bf03-5bc9f17dac97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.826427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0d69e-f802-46ef-9a24-3337a4a8c388 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:54.828875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-787bb00c-55ac-434e-bf03-5bc9f17dac97 None None] MFA Rules not processed for user `e0389958e0ac4690a3d1630a27d2cfd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:54.830160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-787bb00c-55ac-434e-bf03-5bc9f17dac97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:54.876420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000d420-9c12-4bf5-ac79-84acb3571611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:54.877027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000d420-9c12-4bf5-ac79-84acb3571611 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:54.877314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000d420-9c12-4bf5-ac79-84acb3571611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:54.877586 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000d420-9c12-4bf5-ac79-84acb3571611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:54.877718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:54.902340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1030/2047] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:54.906724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787bb00c-55ac-434e-bf03-5bc9f17dac97 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:54.946233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000d420-9c12-4bf5-ac79-84acb3571611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:54.947871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000d420-9c12-4bf5-ac79-84acb3571611 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:54.959318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1018/2048] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:55.005445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787bb00c-55ac-434e-bf03-5bc9f17dac97 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:55.010780 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343273a-d0b8-4859-9e3e-3aea19081ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:55.011497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343273a-d0b8-4859-9e3e-3aea19081ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.011781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343273a-d0b8-4859-9e3e-3aea19081ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.012053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343273a-d0b8-4859-9e3e-3aea19081ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.013005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343273a-d0b8-4859-9e3e-3aea19081ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:55.015633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343273a-d0b8-4859-9e3e-3aea19081ed5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:55.072706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1031/2049] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:54 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:55.073154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000d420-9c12-4bf5-ac79-84acb3571611 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:55.086868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000d420-9c12-4bf5-ac79-84acb3571611 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:55.109480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:55.214095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089e352-a37c-4d76-8537-fa214b592550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:55.215351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089e352-a37c-4d76-8537-fa214b592550 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.215635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089e352-a37c-4d76-8537-fa214b592550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.216446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089e352-a37c-4d76-8537-fa214b592550 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.217146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089e352-a37c-4d76-8537-fa214b592550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:55.219190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089e352-a37c-4d76-8537-fa214b592550 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:55.254446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1019/2050] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:55 2022] POST /identity/v3/users => generated 296 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:55.258707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343273a-d0b8-4859-9e3e-3aea19081ed5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:55.273909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343273a-d0b8-4859-9e3e-3aea19081ed5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:55.280201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361bd67-4c44-462b-b609-4a0e1ee6ba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:55.280911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361bd67-4c44-462b-b609-4a0e1ee6ba1d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.281275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361bd67-4c44-462b-b609-4a0e1ee6ba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.281606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361bd67-4c44-462b-b609-4a0e1ee6ba1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.282323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361bd67-4c44-462b-b609-4a0e1ee6ba1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:55.284155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361bd67-4c44-462b-b609-4a0e1ee6ba1d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:55.296763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1032/2051] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:55.300882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089e352-a37c-4d76-8537-fa214b592550 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:55.311633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089e352-a37c-4d76-8537-fa214b592550 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:55.317109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1b4ba-b74f-47bf-a424-3083f638f32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:55.317888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1b4ba-b74f-47bf-a424-3083f638f32f admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.318875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1b4ba-b74f-47bf-a424-3083f638f32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.319248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1b4ba-b74f-47bf-a424-3083f638f32f admin admin] PATH_INFO: `/v3/projects/aa8e1a2f975b4a3c803102b78a5a95e8/users/b5e6a964158947ca840cc4de8ce6101e/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.338210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1b4ba-b74f-47bf-a424-3083f638f32f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8e1a2f975b4a3c803102b78a5a95e8, user_id=b5e6a964158947ca840cc4de8ce6101e, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:55.341698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1b4ba-b74f-47bf-a424-3083f638f32f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:55.356353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1020/2052] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:55 2022] PUT /identity/v3/projects/aa8e1a2f975b4a3c803102b78a5a95e8/users/b5e6a964158947ca840cc4de8ce6101e/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:55.360843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361bd67-4c44-462b-b609-4a0e1ee6ba1d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:55.371173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361bd67-4c44-462b-b609-4a0e1ee6ba1d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:55.451765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feced75-5fbf-4d59-95fd-d85e03f2efd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:55.452464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feced75-5fbf-4d59-95fd-d85e03f2efd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.452753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feced75-5fbf-4d59-95fd-d85e03f2efd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.453011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feced75-5fbf-4d59-95fd-d85e03f2efd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.453646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feced75-5fbf-4d59-95fd-d85e03f2efd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:55.455248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feced75-5fbf-4d59-95fd-d85e03f2efd9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:55.470829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1033/2053] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:55.475000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1b4ba-b74f-47bf-a424-3083f638f32f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:55.485307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1b4ba-b74f-47bf-a424-3083f638f32f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:55.490306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74830a5d-1ee8-4984-9fc1-b4212d686baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:55.490941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74830a5d-1ee8-4984-9fc1-b4212d686baa admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.491258 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74830a5d-1ee8-4984-9fc1-b4212d686baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.491530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74830a5d-1ee8-4984-9fc1-b4212d686baa admin admin] PATH_INFO: `/v3/projects/aa8e1a2f975b4a3c803102b78a5a95e8/users/b5e6a964158947ca840cc4de8ce6101e/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.493499 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74830a5d-1ee8-4984-9fc1-b4212d686baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8e1a2f975b4a3c803102b78a5a95e8, user_id=b5e6a964158947ca840cc4de8ce6101e, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:55.496432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74830a5d-1ee8-4984-9fc1-b4212d686baa admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:55.507361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1021/2054] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:55 2022] PUT /identity/v3/projects/aa8e1a2f975b4a3c803102b78a5a95e8/users/b5e6a964158947ca840cc4de8ce6101e/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:55.518219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbef2f-d4bb-4af7-80d1-99f6772a22b3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.519202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbef2f-d4bb-4af7-80d1-99f6772a22b3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.519509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbef2f-d4bb-4af7-80d1-99f6772a22b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.547835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-33dbef2f-d4bb-4af7-80d1-99f6772a22b3 None None] MFA Rules not processed for user `b5e6a964158947ca840cc4de8ce6101e`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:55.549052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-33dbef2f-d4bb-4af7-80d1-99f6772a22b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:55.590197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:55.625957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1034/2055] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:55.639420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74830a5d-1ee8-4984-9fc1-b4212d686baa admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:55.641511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-74830a5d-1ee8-4984-9fc1-b4212d686baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:55.705127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74830a5d-1ee8-4984-9fc1-b4212d686baa admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:55.708993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3244d-db6e-4f96-980d-6192a97651ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:55.709605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3244d-db6e-4f96-980d-6192a97651ec service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.709882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3244d-db6e-4f96-980d-6192a97651ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.710280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3244d-db6e-4f96-980d-6192a97651ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.719843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3244d-db6e-4f96-980d-6192a97651ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:55.728787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3244d-db6e-4f96-980d-6192a97651ec service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:55.730192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684dd6c-938b-4a4e-b829-776104cc7f58 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.730482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684dd6c-938b-4a4e-b829-776104cc7f58 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.730757 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684dd6c-938b-4a4e-b829-776104cc7f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.731669 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1022/2056] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:55.788894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d684dd6c-938b-4a4e-b829-776104cc7f58 None None] MFA Rules not processed for user `7ecb55aabbe64be7907123772c359287`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:55.790868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d684dd6c-938b-4a4e-b829-776104cc7f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:55.875725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1035/2057] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:03:55.886748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3244d-db6e-4f96-980d-6192a97651ec service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:55.898275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3244d-db6e-4f96-980d-6192a97651ec service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:55.902749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a765b1-1294-4f17-bce8-04fe22a49c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:55.903518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a765b1-1294-4f17-bce8-04fe22a49c33 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:55.903858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a765b1-1294-4f17-bce8-04fe22a49c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:55.904198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a765b1-1294-4f17-bce8-04fe22a49c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:55.913305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a765b1-1294-4f17-bce8-04fe22a49c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:55.914719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a765b1-1294-4f17-bce8-04fe22a49c33 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:55.924339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1023/2058] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:03:56.098887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684dd6c-938b-4a4e-b829-776104cc7f58 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:56.109777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684dd6c-938b-4a4e-b829-776104cc7f58 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:56.114240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752a74e-a07c-45e5-b011-c4a0acdab23f tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5e6a964158947ca840cc4de8ce6101e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa8e1a2f975b4a3c803102b78a5a95e8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:56.114832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752a74e-a07c-45e5-b011-c4a0acdab23f tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:56.115210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752a74e-a07c-45e5-b011-c4a0acdab23f tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:56.115489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752a74e-a07c-45e5-b011-c4a0acdab23f tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:56.116131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752a74e-a07c-45e5-b011-c4a0acdab23f tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:56.116429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752a74e-a07c-45e5-b011-c4a0acdab23f tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:56.118006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752a74e-a07c-45e5-b011-c4a0acdab23f tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:56.129860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1036/2059] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:56.134068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a765b1-1294-4f17-bce8-04fe22a49c33 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:56.147415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a765b1-1294-4f17-bce8-04fe22a49c33 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:56.152753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d1727-1449-40c3-974d-fb6fd1ecc227 tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5e6a964158947ca840cc4de8ce6101e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa8e1a2f975b4a3c803102b78a5a95e8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:56.154008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d1727-1449-40c3-974d-fb6fd1ecc227 tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:56.154369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d1727-1449-40c3-974d-fb6fd1ecc227 tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:56.154738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d1727-1449-40c3-974d-fb6fd1ecc227 tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:56.155591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d1727-1449-40c3-974d-fb6fd1ecc227 tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:56.157873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d1727-1449-40c3-974d-fb6fd1ecc227 tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:56.211385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1024/2060] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:56 2022] POST /identity/v3/projects => generated 354 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:56.215297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752a74e-a07c-45e5-b011-c4a0acdab23f tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:56.226365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752a74e-a07c-45e5-b011-c4a0acdab23f tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:56.292955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5e6a964158947ca840cc4de8ce6101e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa8e1a2f975b4a3c803102b78a5a95e8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:56.293566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:56.294365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:56.294636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:56.295316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:56.295622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:56.297199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:56.309108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1037/2061] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:56.348858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d1727-1449-40c3-974d-fb6fd1ecc227 tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:56.360068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d1727-1449-40c3-974d-fb6fd1ecc227 tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:56.364750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5e6a964158947ca840cc4de8ce6101e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa8e1a2f975b4a3c803102b78a5a95e8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:56.365376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:56.365659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:56.365932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] PATH_INFO: `/v3/projects/98bc7cf065b54d8db6d171af12f8b87e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:56.367073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Authorizing `identity:delete_project(project_id=98bc7cf065b54d8db6d171af12f8b87e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:56.369027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:56.412417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:56.431146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98bc7cf065b54d8db6d171af12f8b87e', 'request_id': 'req-c91a62ee-3938-4fc6-ae33-046f30d2432c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.438482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98bc7cf065b54d8db6d171af12f8b87e', 'request_id': 'req-c91a62ee-3938-4fc6-ae33-046f30d2432c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.446047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98bc7cf065b54d8db6d171af12f8b87e', 'request_id': 'req-c91a62ee-3938-4fc6-ae33-046f30d2432c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.456476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] The token cache is being invalidate because project 98bc7cf065b54d8db6d171af12f8b87e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:56.456817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.457389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c91a62ee-3938-4fc6-ae33-046f30d2432c tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.458928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1025/2062] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:56 2022] DELETE /identity/v3/projects/98bc7cf065b54d8db6d171af12f8b87e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:56.665704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:56.667454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:56.721674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:56.738084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a31086-cba7-43b2-a6ae-148c6e3ea17d tempest-VolumeQuotaClassesTest-845175707 tempest-VolumeQuotaClassesTest-845175707-project-admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:56.792365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:56.792992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:56.793270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:56.793542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] PATH_INFO: `/v3/users/e0389958e0ac4690a3d1630a27d2cfd1` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:56.795305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0389958e0ac4690a3d1630a27d2cfd1)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:56.797233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:56.845418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0389958e0ac4690a3d1630a27d2cfd1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.859108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0389958e0ac4690a3d1630a27d2cfd1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.865119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0389958e0ac4690a3d1630a27d2cfd1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.865852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0389958e0ac4690a3d1630a27d2cfd1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.866460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0389958e0ac4690a3d1630a27d2cfd1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:56.878616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1038/2063] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:56 2022] DELETE /identity/v3/users/e0389958e0ac4690a3d1630a27d2cfd1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:56.883264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504cc10-e9ba-43c8-84af-7e9b9285f05f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:56.883553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504cc10-e9ba-43c8-84af-7e9b9285f05f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:56.883828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504cc10-e9ba-43c8-84af-7e9b9285f05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:56.902456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-b504cc10-e9ba-43c8-84af-7e9b9285f05f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:56.904248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b504cc10-e9ba-43c8-84af-7e9b9285f05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:56.964393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1026/2064] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:56.975482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:56.976906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:57.039995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c85bd-0484-4a28-aa94-22ced6c141db admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:57.043718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b20c1-2dcf-45e6-b23e-99704cb515bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:57.044756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b20c1-2dcf-45e6-b23e-99704cb515bf service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:57.045059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b20c1-2dcf-45e6-b23e-99704cb515bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:57.045301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b20c1-2dcf-45e6-b23e-99704cb515bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:57.054437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b20c1-2dcf-45e6-b23e-99704cb515bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:57.055937 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b20c1-2dcf-45e6-b23e-99704cb515bf service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:57.065707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1039/2065] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:03:57.338623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeed81-2fd0-4406-b40d-2122cf64438c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:57.340301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeed81-2fd0-4406-b40d-2122cf64438c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:57.341646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeed81-2fd0-4406-b40d-2122cf64438c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:57.384594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-5fbeed81-2fd0-4406-b40d-2122cf64438c None None] MFA Rules not processed for user `7ecb55aabbe64be7907123772c359287`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:57.388431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5fbeed81-2fd0-4406-b40d-2122cf64438c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:57.473224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b20c1-2dcf-45e6-b23e-99704cb515bf service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:57.475472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a49b20c1-2dcf-45e6-b23e-99704cb515bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:57.509013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1027/2066] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:57.518926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b20c1-2dcf-45e6-b23e-99704cb515bf service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:57.527505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:57.528723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:57.529343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:57.529872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] PATH_INFO: `/v3/projects/1231cc02be374a239d73daba2eaf4a75` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:57.531236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=1231cc02be374a239d73daba2eaf4a75)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:57.536451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:57.556725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:57.639707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1231cc02be374a239d73daba2eaf4a75', 'request_id': 'req-bb01eee9-7360-4588-bc51-1232859e7dde'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.650698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1231cc02be374a239d73daba2eaf4a75', 'request_id': 'req-bb01eee9-7360-4588-bc51-1232859e7dde'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.665570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1231cc02be374a239d73daba2eaf4a75', 'request_id': 'req-bb01eee9-7360-4588-bc51-1232859e7dde'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.681683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] The token cache is being invalidate because project 1231cc02be374a239d73daba2eaf4a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:57.682401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.683971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.689406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1040/2067] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:57 2022] DELETE /identity/v3/projects/1231cc02be374a239d73daba2eaf4a75 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:57.690708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeed81-2fd0-4406-b40d-2122cf64438c None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:57.702790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeed81-2fd0-4406-b40d-2122cf64438c None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:57.786880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:57.787614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:57.787897 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:57.788171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] PATH_INFO: `/v3/users/b5e6a964158947ca840cc4de8ce6101e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:57.789310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e6a964158947ca840cc4de8ce6101e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:57.791500 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:57.795002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:57.857229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e6a964158947ca840cc4de8ce6101e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.868069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e6a964158947ca840cc4de8ce6101e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.876725 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e6a964158947ca840cc4de8ce6101e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.896393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e6a964158947ca840cc4de8ce6101e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.897710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b1b46299-4408-44d5-9a56-068f4a6d1fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e6a964158947ca840cc4de8ce6101e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:57.904762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1028/2068] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:03:57 2022] DELETE /identity/v3/users/b5e6a964158947ca840cc4de8ce6101e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:03:58.226189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:58.227888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:58.246804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01eee9-7360-4588-bc51-1232859e7dde admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:58.312940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:58.313688 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:58.314034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:58.314366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] PATH_INFO: `/v3/projects/aa8e1a2f975b4a3c803102b78a5a95e8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:58.315675 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8e1a2f975b4a3c803102b78a5a95e8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:58.317915 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:58.408522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8e1a2f975b4a3c803102b78a5a95e8', 'request_id': 'req-731226a1-a6fb-4fe1-b84e-f690fab04f0b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:58.425046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8e1a2f975b4a3c803102b78a5a95e8', 'request_id': 'req-731226a1-a6fb-4fe1-b84e-f690fab04f0b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:58.433668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8e1a2f975b4a3c803102b78a5a95e8', 'request_id': 'req-731226a1-a6fb-4fe1-b84e-f690fab04f0b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:58.445622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] The token cache is being invalidate because project aa8e1a2f975b4a3c803102b78a5a95e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:03:58.446022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:58.446800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-731226a1-a6fb-4fe1-b84e-f690fab04f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:03:58.448625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1041/2069] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:03:58 2022] DELETE /identity/v3/projects/aa8e1a2f975b4a3c803102b78a5a95e8 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:58.454227 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aa840-b266-43f4-a703-2e9fc6a086a3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:58.455427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aa840-b266-43f4-a703-2e9fc6a086a3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:58.455709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aa840-b266-43f4-a703-2e9fc6a086a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:58.478414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-772aa840-b266-43f4-a703-2e9fc6a086a3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:58.481979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-772aa840-b266-43f4-a703-2e9fc6a086a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:58.566870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1029/2070] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:58.576724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095400a3-f3fd-4410-85fd-0021e583f165 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:58.577135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095400a3-f3fd-4410-85fd-0021e583f165 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:58.577594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095400a3-f3fd-4410-85fd-0021e583f165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:58.614059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-095400a3-f3fd-4410-85fd-0021e583f165 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:58.615788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-095400a3-f3fd-4410-85fd-0021e583f165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:58.626667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1042/2071] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:58.639423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aa840-b266-43f4-a703-2e9fc6a086a3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:58.678218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aa840-b266-43f4-a703-2e9fc6a086a3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:58.712676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915f35a-0ae4-4bf6-8cdc-f18015a9805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:58.714471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915f35a-0ae4-4bf6-8cdc-f18015a9805e admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:58.715185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915f35a-0ae4-4bf6-8cdc-f18015a9805e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:58.716495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915f35a-0ae4-4bf6-8cdc-f18015a9805e admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:58.718427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915f35a-0ae4-4bf6-8cdc-f18015a9805e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:03:58.719624 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915f35a-0ae4-4bf6-8cdc-f18015a9805e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:58.723693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915f35a-0ae4-4bf6-8cdc-f18015a9805e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:58.741675 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1030/2072] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:03:58.748148 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095400a3-f3fd-4410-85fd-0021e583f165 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:58.772205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095400a3-f3fd-4410-85fd-0021e583f165 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:58.784788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48690e77-924f-4b60-aed1-24fecd72e402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:58.786624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48690e77-924f-4b60-aed1-24fecd72e402 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:58.787145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48690e77-924f-4b60-aed1-24fecd72e402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:58.788020 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48690e77-924f-4b60-aed1-24fecd72e402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:58.789035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48690e77-924f-4b60-aed1-24fecd72e402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:58.791994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48690e77-924f-4b60-aed1-24fecd72e402 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:58.849996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1043/2073] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:58 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:58.853969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915f35a-0ae4-4bf6-8cdc-f18015a9805e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:58.881616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915f35a-0ae4-4bf6-8cdc-f18015a9805e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:59.055015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4045074-01c9-4f93-b3ca-91285e58f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:59.055780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4045074-01c9-4f93-b3ca-91285e58f3ab admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:59.056164 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4045074-01c9-4f93-b3ca-91285e58f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:59.056691 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4045074-01c9-4f93-b3ca-91285e58f3ab admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:59.057722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4045074-01c9-4f93-b3ca-91285e58f3ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:59.060592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4045074-01c9-4f93-b3ca-91285e58f3ab admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:59.110193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:59.120614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1031/2074] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:58 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:59.121716 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48690e77-924f-4b60-aed1-24fecd72e402 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:59.127842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:03:59.135864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48690e77-924f-4b60-aed1-24fecd72e402 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:59.148895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fb581-4741-4982-93a3-af6df10bcbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:59.149613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fb581-4741-4982-93a3-af6df10bcbee admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:59.150860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fb581-4741-4982-93a3-af6df10bcbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:59.151406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fb581-4741-4982-93a3-af6df10bcbee admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:59.152265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fb581-4741-4982-93a3-af6df10bcbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:59.154903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fb581-4741-4982-93a3-af6df10bcbee admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:59.167805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1044/2075] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:59.172352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4045074-01c9-4f93-b3ca-91285e58f3ab admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:59.183662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4045074-01c9-4f93-b3ca-91285e58f3ab admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:59.189978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b47d9-ed42-4506-9dce-045da0da72d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:59.189978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b47d9-ed42-4506-9dce-045da0da72d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:59.189978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b47d9-ed42-4506-9dce-045da0da72d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:59.190302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b47d9-ed42-4506-9dce-045da0da72d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:59.191079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b47d9-ed42-4506-9dce-045da0da72d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:59.192776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b47d9-ed42-4506-9dce-045da0da72d4 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:59.214416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1032/2076] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:03:59.216914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fb581-4741-4982-93a3-af6df10bcbee admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:59.227567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fb581-4741-4982-93a3-af6df10bcbee admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:59.232778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940bcb3-bc4a-4043-9e71-8c7e6c4c39dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:59.233578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940bcb3-bc4a-4043-9e71-8c7e6c4c39dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:59.234066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940bcb3-bc4a-4043-9e71-8c7e6c4c39dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:59.234445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940bcb3-bc4a-4043-9e71-8c7e6c4c39dc admin admin] PATH_INFO: `/v3/projects/aeb70d464ee445ec95a59e1816a6f071/users/d92f2b515a9146ba9257afb840880294/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:59.263130 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940bcb3-bc4a-4043-9e71-8c7e6c4c39dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb70d464ee445ec95a59e1816a6f071, user_id=d92f2b515a9146ba9257afb840880294, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:59.266594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940bcb3-bc4a-4043-9e71-8c7e6c4c39dc admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:59.287572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1045/2077] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:59 2022] PUT /identity/v3/projects/aeb70d464ee445ec95a59e1816a6f071/users/d92f2b515a9146ba9257afb840880294/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:59.299023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542b019-0091-40f3-94b8-8bd488a8176b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:59.299310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542b019-0091-40f3-94b8-8bd488a8176b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:59.299584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542b019-0091-40f3-94b8-8bd488a8176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:59.350422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-2542b019-0091-40f3-94b8-8bd488a8176b None None] MFA Rules not processed for user `d92f2b515a9146ba9257afb840880294`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:03:59.351798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-2542b019-0091-40f3-94b8-8bd488a8176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:03:59.474755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1033/2078] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:03:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:03:59.475690 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940bcb3-bc4a-4043-9e71-8c7e6c4c39dc admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:59.603491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940bcb3-bc4a-4043-9e71-8c7e6c4c39dc admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:59.609347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709c80c-4c65-42ab-b72b-bdbc6c458702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:59.610314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709c80c-4c65-42ab-b72b-bdbc6c458702 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:59.610742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709c80c-4c65-42ab-b72b-bdbc6c458702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:59.611202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709c80c-4c65-42ab-b72b-bdbc6c458702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:59.612023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709c80c-4c65-42ab-b72b-bdbc6c458702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:59.614112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709c80c-4c65-42ab-b72b-bdbc6c458702 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:59.648731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1046/2079] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:03:59 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:03:59.652889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542b019-0091-40f3-94b8-8bd488a8176b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:59.669037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542b019-0091-40f3-94b8-8bd488a8176b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:59.737242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcec27e-d403-424d-9efc-12538b732dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:59.737880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcec27e-d403-424d-9efc-12538b732dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:59.738159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcec27e-d403-424d-9efc-12538b732dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:59.738451 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcec27e-d403-424d-9efc-12538b732dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:59.739112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcec27e-d403-424d-9efc-12538b732dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:59.741143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcec27e-d403-424d-9efc-12538b732dd2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:59.769033 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1034/2080] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:03:59 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:03:59.771132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709c80c-4c65-42ab-b72b-bdbc6c458702 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:59.783656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709c80c-4c65-42ab-b72b-bdbc6c458702 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:59.803751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37018ed-a518-434b-9df6-6cdc2a7817bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:59.804582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37018ed-a518-434b-9df6-6cdc2a7817bf admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:59.805059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37018ed-a518-434b-9df6-6cdc2a7817bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:59.805355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37018ed-a518-434b-9df6-6cdc2a7817bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:59.806022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37018ed-a518-434b-9df6-6cdc2a7817bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:59.807667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37018ed-a518-434b-9df6-6cdc2a7817bf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:59.831629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1047/2081] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:03:59.838665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcec27e-d403-424d-9efc-12538b732dd2 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:59.855453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcec27e-d403-424d-9efc-12538b732dd2 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:03:59.862239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7050-f2e3-43f7-9d23-8708481dd708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:03:59.863122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7050-f2e3-43f7-9d23-8708481dd708 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:03:59.863521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7050-f2e3-43f7-9d23-8708481dd708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:03:59.863854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7050-f2e3-43f7-9d23-8708481dd708 admin admin] PATH_INFO: `/v3/projects/87e1ed1a30dd4743b06457cbfb966de8/users/5ca4fc82aaa24635915024c3cda492bf/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:03:59.881386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf7050-f2e3-43f7-9d23-8708481dd708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e1ed1a30dd4743b06457cbfb966de8, user_id=5ca4fc82aaa24635915024c3cda492bf, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:03:59.884919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf7050-f2e3-43f7-9d23-8708481dd708 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:03:59.905882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1035/2082] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:03:59 2022] PUT /identity/v3/projects/87e1ed1a30dd4743b06457cbfb966de8/users/5ca4fc82aaa24635915024c3cda492bf/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:03:59.910829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37018ed-a518-434b-9df6-6cdc2a7817bf admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:03:59.922752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37018ed-a518-434b-9df6-6cdc2a7817bf admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:00.006033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d203fe-89eb-4f63-bde1-33f369516fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:00.006662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d203fe-89eb-4f63-bde1-33f369516fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:00.007023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d203fe-89eb-4f63-bde1-33f369516fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:00.007303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d203fe-89eb-4f63-bde1-33f369516fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:00.008222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d203fe-89eb-4f63-bde1-33f369516fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:00.011917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d203fe-89eb-4f63-bde1-33f369516fc8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:00.024152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1048/2083] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:00.028304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7050-f2e3-43f7-9d23-8708481dd708 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:00.048557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7050-f2e3-43f7-9d23-8708481dd708 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:00.062413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66f810-2732-4d66-a2bb-59e88f967c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:00.071231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66f810-2732-4d66-a2bb-59e88f967c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:00.072743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66f810-2732-4d66-a2bb-59e88f967c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:00.074175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66f810-2732-4d66-a2bb-59e88f967c12 admin admin] PATH_INFO: `/v3/projects/87e1ed1a30dd4743b06457cbfb966de8/users/5ca4fc82aaa24635915024c3cda492bf/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:00.081483 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66f810-2732-4d66-a2bb-59e88f967c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e1ed1a30dd4743b06457cbfb966de8, user_id=5ca4fc82aaa24635915024c3cda492bf, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:00.088294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66f810-2732-4d66-a2bb-59e88f967c12 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:00.107840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1036/2084] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:00 2022] PUT /identity/v3/projects/87e1ed1a30dd4743b06457cbfb966de8/users/5ca4fc82aaa24635915024c3cda492bf/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:00.120150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f674664-2b26-4a84-92b4-ddee406fbd13 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:00.120537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f674664-2b26-4a84-92b4-ddee406fbd13 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:00.120873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f674664-2b26-4a84-92b4-ddee406fbd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:00.171343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5f674664-2b26-4a84-92b4-ddee406fbd13 None None] MFA Rules not processed for user `5ca4fc82aaa24635915024c3cda492bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:00.173021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5f674664-2b26-4a84-92b4-ddee406fbd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:00.206846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:00.271998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1049/2085] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:00.284303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66f810-2732-4d66-a2bb-59e88f967c12 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:00.286688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7c66f810-2732-4d66-a2bb-59e88f967c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:00.324878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:00.360687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66f810-2732-4d66-a2bb-59e88f967c12 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:00.364657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9e24b-d178-4916-8981-dbb6a9df0543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:00.365369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9e24b-d178-4916-8981-dbb6a9df0543 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:00.365739 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9e24b-d178-4916-8981-dbb6a9df0543 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:00.366116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9e24b-d178-4916-8981-dbb6a9df0543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:00.377380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9e24b-d178-4916-8981-dbb6a9df0543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:00.379015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9e24b-d178-4916-8981-dbb6a9df0543 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:00.391131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1037/2086] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:01.691778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f674664-2b26-4a84-92b4-ddee406fbd13 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:01.772417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f674664-2b26-4a84-92b4-ddee406fbd13 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:01.784468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:01.784468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:01.784468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:01.784468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] PATH_INFO: `/v3/users/d92f2b515a9146ba9257afb840880294` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:01.795981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d92f2b515a9146ba9257afb840880294)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:01.798150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:01.895707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd92f2b515a9146ba9257afb840880294'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:01.911709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd92f2b515a9146ba9257afb840880294'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:01.948781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd92f2b515a9146ba9257afb840880294'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:01.951343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd92f2b515a9146ba9257afb840880294'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:01.952101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd92f2b515a9146ba9257afb840880294'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:01.973724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1050/2087] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:01 2022] DELETE /identity/v3/users/d92f2b515a9146ba9257afb840880294 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:01.978186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e6bf-df44-48eb-8f5f-4bd4da6d843c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:01.978636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e6bf-df44-48eb-8f5f-4bd4da6d843c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:01.978858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e6bf-df44-48eb-8f5f-4bd4da6d843c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:02.023598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f6d5e6bf-df44-48eb-8f5f-4bd4da6d843c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:02.026020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f6d5e6bf-df44-48eb-8f5f-4bd4da6d843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:02.104394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1038/2088] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:02.134773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:02.137270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:02.188783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5e6bf-df44-48eb-8f5f-4bd4da6d843c None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:02.190235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f6d5e6bf-df44-48eb-8f5f-4bd4da6d843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:02.211382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5e6bf-df44-48eb-8f5f-4bd4da6d843c None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:02.227208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:02.232218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:02.232916 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:02.233237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:02.233549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] PATH_INFO: `/v3/users/9488b5c821a244ddb49407cb2f53c651` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:02.234862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9488b5c821a244ddb49407cb2f53c651)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:02.236766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:02.246483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914491ac-72b4-458d-8085-f51dd945e985 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:02.258742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a37723-3e01-4dc5-933a-6a72387c67b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:02.259550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a37723-3e01-4dc5-933a-6a72387c67b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:02.259830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a37723-3e01-4dc5-933a-6a72387c67b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:02.260103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a37723-3e01-4dc5-933a-6a72387c67b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:02.289504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a37723-3e01-4dc5-933a-6a72387c67b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:02.292337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a37723-3e01-4dc5-933a-6a72387c67b8 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:02.321520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1051/2089] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:02.344519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9488b5c821a244ddb49407cb2f53c651'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:02.357322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9488b5c821a244ddb49407cb2f53c651'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:02.369962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9488b5c821a244ddb49407cb2f53c651'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:02.420093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9488b5c821a244ddb49407cb2f53c651'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:02.420955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9488b5c821a244ddb49407cb2f53c651'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:02.444579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1039/2090] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:02 2022] DELETE /identity/v3/users/9488b5c821a244ddb49407cb2f53c651 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:02.449401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e105d-2bb5-4c2a-8ef8-9ba98bf45a31 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:02.449692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e105d-2bb5-4c2a-8ef8-9ba98bf45a31 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:02.449997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e105d-2bb5-4c2a-8ef8-9ba98bf45a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:02.489254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c28e105d-2bb5-4c2a-8ef8-9ba98bf45a31 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:02.494100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c28e105d-2bb5-4c2a-8ef8-9ba98bf45a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:02.691057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1052/2091] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:02.707825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:02.709787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:02.821539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:02.839926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b733-9871-401f-8ba3-433a78556fe4 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:02.845937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6693f8-bea3-4e9d-aef0-bec55ac9efc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:02.846880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6693f8-bea3-4e9d-aef0-bec55ac9efc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:02.847463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6693f8-bea3-4e9d-aef0-bec55ac9efc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:02.848089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6693f8-bea3-4e9d-aef0-bec55ac9efc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:02.865341 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6693f8-bea3-4e9d-aef0-bec55ac9efc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:02.867582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6693f8-bea3-4e9d-aef0-bec55ac9efc9 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:02.904459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1040/2092] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:03.238575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e105d-2bb5-4c2a-8ef8-9ba98bf45a31 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:03.240914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c28e105d-2bb5-4c2a-8ef8-9ba98bf45a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:03.279827 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e105d-2bb5-4c2a-8ef8-9ba98bf45a31 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:03.294469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:03.296532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:03.296878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:03.297585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] PATH_INFO: `/v3/projects/aeb70d464ee445ec95a59e1816a6f071` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:03.301461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb70d464ee445ec95a59e1816a6f071)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:03.309391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:03.386752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb70d464ee445ec95a59e1816a6f071', 'request_id': 'req-41ce208c-f8a8-4a32-b443-0b0488213169'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.396723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb70d464ee445ec95a59e1816a6f071', 'request_id': 'req-41ce208c-f8a8-4a32-b443-0b0488213169'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.416435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb70d464ee445ec95a59e1816a6f071', 'request_id': 'req-41ce208c-f8a8-4a32-b443-0b0488213169'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.421798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6693f8-bea3-4e9d-aef0-bec55ac9efc9 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:03.423473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4b6693f8-bea3-4e9d-aef0-bec55ac9efc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:03.431400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] The token cache is being invalidate because project aeb70d464ee445ec95a59e1816a6f071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:03.431854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.433024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.436689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1053/2093] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:03 2022] DELETE /identity/v3/projects/aeb70d464ee445ec95a59e1816a6f071 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:03.439692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:03.441127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:03.442121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6693f8-bea3-4e9d-aef0-bec55ac9efc9 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:03.520743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:03.521540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:03.521697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:03.522021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] PATH_INFO: `/v3/projects/3a860dde44aa4e7f86ddd57055bece30` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:03.523285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a860dde44aa4e7f86ddd57055bece30)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:03.525261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:03.533449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce208c-f8a8-4a32-b443-0b0488213169 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:03.540323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:03.541016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:03.541300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:03.541629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] PATH_INFO: `/v3/users/5ca4fc82aaa24635915024c3cda492bf` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:03.542794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca4fc82aaa24635915024c3cda492bf)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:03.544979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:03.559161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:03.585629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a860dde44aa4e7f86ddd57055bece30', 'request_id': 'req-afb0a518-e535-4928-b0f0-8a91604f28f9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.595408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a860dde44aa4e7f86ddd57055bece30', 'request_id': 'req-afb0a518-e535-4928-b0f0-8a91604f28f9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.603687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a860dde44aa4e7f86ddd57055bece30', 'request_id': 'req-afb0a518-e535-4928-b0f0-8a91604f28f9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.612323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca4fc82aaa24635915024c3cda492bf'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.616652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] The token cache is being invalidate because project 3a860dde44aa4e7f86ddd57055bece30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:03.617012 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.617847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.623062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca4fc82aaa24635915024c3cda492bf'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.623212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1041/2094] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:03 2022] DELETE /identity/v3/projects/3a860dde44aa4e7f86ddd57055bece30 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:03.624756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:03.627366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:03.639546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca4fc82aaa24635915024c3cda492bf'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.640267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca4fc82aaa24635915024c3cda492bf'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.640916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca4fc82aaa24635915024c3cda492bf'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.656245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1054/2095] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:03 2022] DELETE /identity/v3/users/5ca4fc82aaa24635915024c3cda492bf => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:03.700460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0a518-e535-4928-b0f0-8a91604f28f9 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:03.724443 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:03.772088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:03.772721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:03.772999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:03.773309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] PATH_INFO: `/v3/users/2b1a8cb417184b43bb446655a681736a` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:03.774616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1a8cb417184b43bb446655a681736a)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:03.776774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:03.837131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1a8cb417184b43bb446655a681736a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.846837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1a8cb417184b43bb446655a681736a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.855931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1a8cb417184b43bb446655a681736a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.871848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1a8cb417184b43bb446655a681736a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.873498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8b2e5080-a9f3-4b24-b03f-233ac95f973e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1a8cb417184b43bb446655a681736a'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:03.883292 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1042/2096] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:03 2022] DELETE /identity/v3/users/2b1a8cb417184b43bb446655a681736a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:04.126669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:04.128294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:04.143445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2f144-4f5c-4d1f-bbf1-760a0d026a17 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:04.263769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:04.264467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.264666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.265512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] PATH_INFO: `/v3/projects/87e1ed1a30dd4743b06457cbfb966de8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.266897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e1ed1a30dd4743b06457cbfb966de8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:04.269818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:04.366206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e1ed1a30dd4743b06457cbfb966de8', 'request_id': 'req-71deb844-eed5-4c0d-84ad-a35eca7dabae'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.393851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e1ed1a30dd4743b06457cbfb966de8', 'request_id': 'req-71deb844-eed5-4c0d-84ad-a35eca7dabae'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.419285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e1ed1a30dd4743b06457cbfb966de8', 'request_id': 'req-71deb844-eed5-4c0d-84ad-a35eca7dabae'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.437692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] The token cache is being invalidate because project 87e1ed1a30dd4743b06457cbfb966de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:04.438266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.438978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.445452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31179434-cbc5-454f-ab0a-2173c81c52e7 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.445968 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31179434-cbc5-454f-ab0a-2173c81c52e7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.446138 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31179434-cbc5-454f-ab0a-2173c81c52e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.449028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1055/2097] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:04 2022] DELETE /identity/v3/projects/87e1ed1a30dd4743b06457cbfb966de8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:04.466762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-31179434-cbc5-454f-ab0a-2173c81c52e7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:04.468188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-31179434-cbc5-454f-ab0a-2173c81c52e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:04.523554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:04.525967 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:04.568343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1043/2098] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:04.579323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6af6d-c332-4018-b04e-d25e15ef2e32 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.579756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6af6d-c332-4018-b04e-d25e15ef2e32 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.580320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6af6d-c332-4018-b04e-d25e15ef2e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.581475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:04.616414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-eaf6af6d-c332-4018-b04e-d25e15ef2e32 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:04.619271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-eaf6af6d-c332-4018-b04e-d25e15ef2e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:04.647121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1044/2099] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:04.651927 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6af6d-c332-4018-b04e-d25e15ef2e32 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:04.658869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71deb844-eed5-4c0d-84ad-a35eca7dabae admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:04.665724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:04.666939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.667439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.667808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] PATH_INFO: `/v3/projects/e50b6fbedfd74b79b4e4fb0f099e24a0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.670074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50b6fbedfd74b79b4e4fb0f099e24a0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:04.672640 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:04.680507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6af6d-c332-4018-b04e-d25e15ef2e32 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:04.692728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750388c-4830-45ab-9ca9-befb08bf0b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:04.694072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750388c-4830-45ab-9ca9-befb08bf0b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.694576 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750388c-4830-45ab-9ca9-befb08bf0b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.695014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750388c-4830-45ab-9ca9-befb08bf0b92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.695837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750388c-4830-45ab-9ca9-befb08bf0b92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:04:04.699012 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750388c-4830-45ab-9ca9-befb08bf0b92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:04.702638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750388c-4830-45ab-9ca9-befb08bf0b92 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:04.726249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1045/2100] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:04:04.730221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750388c-4830-45ab-9ca9-befb08bf0b92 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:04.744946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750388c-4830-45ab-9ca9-befb08bf0b92 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:04.758807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50b6fbedfd74b79b4e4fb0f099e24a0', 'request_id': 'req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.766837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50b6fbedfd74b79b4e4fb0f099e24a0', 'request_id': 'req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.767376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:04.776837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50b6fbedfd74b79b4e4fb0f099e24a0', 'request_id': 'req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.790198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] The token cache is being invalidate because project e50b6fbedfd74b79b4e4fb0f099e24a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:04.790490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.791180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dd8baa7c-56d3-4c8e-9ed5-4f539c2191c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:04.792784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1056/2101] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:04 2022] DELETE /identity/v3/projects/e50b6fbedfd74b79b4e4fb0f099e24a0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:04.800652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccafcd-118d-4397-a5a7-4583451eaf4b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.801034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccafcd-118d-4397-a5a7-4583451eaf4b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.801317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccafcd-118d-4397-a5a7-4583451eaf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.813641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0efd3a-3c67-4cae-8321-76d3e37a4719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:04.814409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0efd3a-3c67-4cae-8321-76d3e37a4719 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.814790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0efd3a-3c67-4cae-8321-76d3e37a4719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.815198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0efd3a-3c67-4cae-8321-76d3e37a4719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.815929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0efd3a-3c67-4cae-8321-76d3e37a4719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:04.817940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0efd3a-3c67-4cae-8321-76d3e37a4719 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:04.824437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-66ccafcd-118d-4397-a5a7-4583451eaf4b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:04.825692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-66ccafcd-118d-4397-a5a7-4583451eaf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:04.835208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1057/2102] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:04.845056 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9135f-d649-4b39-9bf1-154a58342d3b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.845627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9135f-d649-4b39-9bf1-154a58342d3b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.846145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9135f-d649-4b39-9bf1-154a58342d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.861100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1046/2103] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:04 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:04:04.862340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0efd3a-3c67-4cae-8321-76d3e37a4719 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:04.864037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8f0efd3a-3c67-4cae-8321-76d3e37a4719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:04.866662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1bc9135f-d649-4b39-9bf1-154a58342d3b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:04.867864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1bc9135f-d649-4b39-9bf1-154a58342d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:04.929647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1058/2104] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:04.934079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0efd3a-3c67-4cae-8321-76d3e37a4719 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:04.935968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9135f-d649-4b39-9bf1-154a58342d3b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:04.939574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d5188-328a-4b48-9857-b405a8153440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:04.940189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d5188-328a-4b48-9857-b405a8153440 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.940461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d5188-328a-4b48-9857-b405a8153440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.940735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d5188-328a-4b48-9857-b405a8153440 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.941474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d5188-328a-4b48-9857-b405a8153440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:04.943436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d5188-328a-4b48-9857-b405a8153440 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:04.946663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9135f-d649-4b39-9bf1-154a58342d3b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:04.952166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51825a6-8a67-4b18-a4ed-fef095539f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:04.953488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51825a6-8a67-4b18-a4ed-fef095539f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.953488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51825a6-8a67-4b18-a4ed-fef095539f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.953488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51825a6-8a67-4b18-a4ed-fef095539f11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.953994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51825a6-8a67-4b18-a4ed-fef095539f11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:04:04.954301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51825a6-8a67-4b18-a4ed-fef095539f11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:04.955925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51825a6-8a67-4b18-a4ed-fef095539f11 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:04.965809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1047/2105] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:04 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:04:04.969822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d5188-328a-4b48-9857-b405a8153440 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:04.974040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1059/2106] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:04:04.978702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51825a6-8a67-4b18-a4ed-fef095539f11 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:04.980364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d5188-328a-4b48-9857-b405a8153440 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:04.985303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91b518-d8f8-460f-bcd3-7c5ef2cb9e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:04.985893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91b518-d8f8-460f-bcd3-7c5ef2cb9e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.986168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91b518-d8f8-460f-bcd3-7c5ef2cb9e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.986432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91b518-d8f8-460f-bcd3-7c5ef2cb9e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.987090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91b518-d8f8-460f-bcd3-7c5ef2cb9e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:04.988667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91b518-d8f8-460f-bcd3-7c5ef2cb9e62 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:04.988886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51825a6-8a67-4b18-a4ed-fef095539f11 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:04.993564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54c0c3-2ade-4d57-a9b2-afa0f2f73779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:04.994152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54c0c3-2ade-4d57-a9b2-afa0f2f73779 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:04.994434 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54c0c3-2ade-4d57-a9b2-afa0f2f73779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:04.994996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54c0c3-2ade-4d57-a9b2-afa0f2f73779 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:04.995446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54c0c3-2ade-4d57-a9b2-afa0f2f73779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:04.999673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1048/2107] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:05.003605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91b518-d8f8-460f-bcd3-7c5ef2cb9e62 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.004604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54c0c3-2ade-4d57-a9b2-afa0f2f73779 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.015543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91b518-d8f8-460f-bcd3-7c5ef2cb9e62 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.021411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8107d-f951-471d-8f0a-972f0fdf5633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.022015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8107d-f951-471d-8f0a-972f0fdf5633 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.022419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8107d-f951-471d-8f0a-972f0fdf5633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.022690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8107d-f951-471d-8f0a-972f0fdf5633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.023382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8107d-f951-471d-8f0a-972f0fdf5633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.024964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8107d-f951-471d-8f0a-972f0fdf5633 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.037361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1049/2108] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:05.038802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1060/2109] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:04 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:04:05.042558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8107d-f951-471d-8f0a-972f0fdf5633 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.043861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54c0c3-2ade-4d57-a9b2-afa0f2f73779 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.052889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8107d-f951-471d-8f0a-972f0fdf5633 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.053517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:05.113482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54c0c3-2ade-4d57-a9b2-afa0f2f73779 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.114205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d572e-32b6-4900-bcab-fb3c5010fa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.114860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d572e-32b6-4900-bcab-fb3c5010fa7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.115169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d572e-32b6-4900-bcab-fb3c5010fa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.115442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d572e-32b6-4900-bcab-fb3c5010fa7f admin admin] PATH_INFO: `/v3/projects/25acd9dcf66d40078dd7b6614d8803e1/users/6b7b3cb86eab456fa239bc61fe7ba289/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.118284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe49f5-d45b-41e2-9f23-eefb3437bd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.118975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe49f5-d45b-41e2-9f23-eefb3437bd98 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.119236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe49f5-d45b-41e2-9f23-eefb3437bd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.119517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe49f5-d45b-41e2-9f23-eefb3437bd98 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.120136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe49f5-d45b-41e2-9f23-eefb3437bd98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.122681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe49f5-d45b-41e2-9f23-eefb3437bd98 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.130478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d572e-32b6-4900-bcab-fb3c5010fa7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25acd9dcf66d40078dd7b6614d8803e1, user_id=6b7b3cb86eab456fa239bc61fe7ba289, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.133169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d572e-32b6-4900-bcab-fb3c5010fa7f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.142985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:05.146188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1061/2110] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:05 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:05.150225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe49f5-d45b-41e2-9f23-eefb3437bd98 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.152015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1050/2111] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:05 2022] PUT /identity/v3/projects/25acd9dcf66d40078dd7b6614d8803e1/users/6b7b3cb86eab456fa239bc61fe7ba289/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:05.160888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe49f5-d45b-41e2-9f23-eefb3437bd98 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.162903 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39fec-1596-427e-a263-43b407f6ffe8 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.163226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39fec-1596-427e-a263-43b407f6ffe8 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.163504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39fec-1596-427e-a263-43b407f6ffe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.191747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-62c39fec-1596-427e-a263-43b407f6ffe8 None None] MFA Rules not processed for user `6b7b3cb86eab456fa239bc61fe7ba289`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:05.193108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-62c39fec-1596-427e-a263-43b407f6ffe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:05.226709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716597-9f2c-4d47-b56a-e9df1deee2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.227351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716597-9f2c-4d47-b56a-e9df1deee2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.227628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716597-9f2c-4d47-b56a-e9df1deee2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.227903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716597-9f2c-4d47-b56a-e9df1deee2e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.228660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f716597-9f2c-4d47-b56a-e9df1deee2e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.230257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f716597-9f2c-4d47-b56a-e9df1deee2e1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.239901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1062/2112] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:05.243739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716597-9f2c-4d47-b56a-e9df1deee2e1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.254727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716597-9f2c-4d47-b56a-e9df1deee2e1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.258215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1051/2113] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:05.261413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9094b9-0c05-4cb8-b943-0cb1fcc8383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.262092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9094b9-0c05-4cb8-b943-0cb1fcc8383c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.263106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9094b9-0c05-4cb8-b943-0cb1fcc8383c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.263234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39fec-1596-427e-a263-43b407f6ffe8 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.263498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9094b9-0c05-4cb8-b943-0cb1fcc8383c admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.264284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9094b9-0c05-4cb8-b943-0cb1fcc8383c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.265858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9094b9-0c05-4cb8-b943-0cb1fcc8383c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.274189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39fec-1596-427e-a263-43b407f6ffe8 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.277098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1063/2114] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:05.280210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8204a7-5f1b-4756-b96e-b12f6a4e7122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.280884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8204a7-5f1b-4756-b96e-b12f6a4e7122 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.281205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9094b9-0c05-4cb8-b943-0cb1fcc8383c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.281310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8204a7-5f1b-4756-b96e-b12f6a4e7122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.281590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8204a7-5f1b-4756-b96e-b12f6a4e7122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.282284 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8204a7-5f1b-4756-b96e-b12f6a4e7122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.284189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8204a7-5f1b-4756-b96e-b12f6a4e7122 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.292707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9094b9-0c05-4cb8-b943-0cb1fcc8383c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.298517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec51307-856a-4155-bea9-2f4ee1ddbbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.299390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec51307-856a-4155-bea9-2f4ee1ddbbae admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.299814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec51307-856a-4155-bea9-2f4ee1ddbbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.300233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec51307-856a-4155-bea9-2f4ee1ddbbae admin admin] PATH_INFO: `/v3/projects/d55d42ab558d46758144adae5e0b4193/users/4021cfdf4e0040e387314c4ca0bc62f6/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.314098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1052/2115] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:05 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:05.318213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8204a7-5f1b-4756-b96e-b12f6a4e7122 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.318630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec51307-856a-4155-bea9-2f4ee1ddbbae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55d42ab558d46758144adae5e0b4193, user_id=4021cfdf4e0040e387314c4ca0bc62f6, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.321508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec51307-856a-4155-bea9-2f4ee1ddbbae admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.339504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1064/2116] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:05 2022] PUT /identity/v3/projects/d55d42ab558d46758144adae5e0b4193/users/4021cfdf4e0040e387314c4ca0bc62f6/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:05.358322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb605b1c-cded-4363-85c9-12493204c86d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.359260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb605b1c-cded-4363-85c9-12493204c86d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.359791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb605b1c-cded-4363-85c9-12493204c86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.401273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8204a7-5f1b-4756-b96e-b12f6a4e7122 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.404565 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cb605b1c-cded-4363-85c9-12493204c86d None None] MFA Rules not processed for user `4021cfdf4e0040e387314c4ca0bc62f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:05.406024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cb605b1c-cded-4363-85c9-12493204c86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:05.483419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0566f4-a8ac-43b0-b5ff-637605b7d9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.484454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0566f4-a8ac-43b0-b5ff-637605b7d9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.484878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0566f4-a8ac-43b0-b5ff-637605b7d9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.485381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0566f4-a8ac-43b0-b5ff-637605b7d9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.486208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0566f4-a8ac-43b0-b5ff-637605b7d9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.488248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0566f4-a8ac-43b0-b5ff-637605b7d9c8 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.496017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:05.514335 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1053/2117] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:05 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:05.518204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0566f4-a8ac-43b0-b5ff-637605b7d9c8 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.522257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1065/2118] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:05.527007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:05.534877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0566f4-a8ac-43b0-b5ff-637605b7d9c8 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.535061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb605b1c-cded-4363-85c9-12493204c86d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.536644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cb605b1c-cded-4363-85c9-12493204c86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:05.541448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42e5ca-ea01-438a-95fc-d15066cdb26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.542136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42e5ca-ea01-438a-95fc-d15066cdb26c admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.542507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42e5ca-ea01-438a-95fc-d15066cdb26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.542855 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42e5ca-ea01-438a-95fc-d15066cdb26c admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.543628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42e5ca-ea01-438a-95fc-d15066cdb26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.545849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42e5ca-ea01-438a-95fc-d15066cdb26c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.569491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1054/2119] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:05.571739 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42e5ca-ea01-438a-95fc-d15066cdb26c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.582823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42e5ca-ea01-438a-95fc-d15066cdb26c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.587945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c724c9-67f5-41f4-9de6-aadc0a5a81ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.588623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c724c9-67f5-41f4-9de6-aadc0a5a81ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.588901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c724c9-67f5-41f4-9de6-aadc0a5a81ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.589173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c724c9-67f5-41f4-9de6-aadc0a5a81ac admin admin] PATH_INFO: `/v3/projects/14936c78981c47abadc357c2b8b66844/users/c6405a9c349a4254aea3023dd5880169/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.597357 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb605b1c-cded-4363-85c9-12493204c86d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.601723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1583cc7-4d83-4e91-a709-2481ffd0d8d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.602333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1583cc7-4d83-4e91-a709-2481ffd0d8d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.602617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1583cc7-4d83-4e91-a709-2481ffd0d8d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.602894 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1583cc7-4d83-4e91-a709-2481ffd0d8d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.604385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c724c9-67f5-41f4-9de6-aadc0a5a81ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=14936c78981c47abadc357c2b8b66844, user_id=c6405a9c349a4254aea3023dd5880169, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.607905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c724c9-67f5-41f4-9de6-aadc0a5a81ac admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.612809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1583cc7-4d83-4e91-a709-2481ffd0d8d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.614635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1583cc7-4d83-4e91-a709-2481ffd0d8d9 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.620640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1055/2120] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:05 2022] PUT /identity/v3/projects/14936c78981c47abadc357c2b8b66844/users/c6405a9c349a4254aea3023dd5880169/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:05.624257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1066/2121] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:04:05.624491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c724c9-67f5-41f4-9de6-aadc0a5a81ac admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.644251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1583cc7-4d83-4e91-a709-2481ffd0d8d9 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.655173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1583cc7-4d83-4e91-a709-2481ffd0d8d9 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.690814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c724c9-67f5-41f4-9de6-aadc0a5a81ac admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.695592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc73360-5be3-45f5-9c0c-51f6b4f018cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.696188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc73360-5be3-45f5-9c0c-51f6b4f018cf admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.696462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc73360-5be3-45f5-9c0c-51f6b4f018cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.696726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc73360-5be3-45f5-9c0c-51f6b4f018cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.697353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc73360-5be3-45f5-9c0c-51f6b4f018cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.698936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc73360-5be3-45f5-9c0c-51f6b4f018cf admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.709363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1056/2122] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:05.713467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc73360-5be3-45f5-9c0c-51f6b4f018cf admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.722774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.723504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.723786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.724066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] PATH_INFO: `/v3/users/4021cfdf4e0040e387314c4ca0bc62f6` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.724686 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc73360-5be3-45f5-9c0c-51f6b4f018cf admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.725238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4021cfdf4e0040e387314c4ca0bc62f6)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.727190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.729456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ce503-6a8a-46f9-a49d-9cb9112b85f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.730115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ce503-6a8a-46f9-a49d-9cb9112b85f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.730377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ce503-6a8a-46f9-a49d-9cb9112b85f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.730709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ce503-6a8a-46f9-a49d-9cb9112b85f5 admin admin] PATH_INFO: `/v3/projects/14936c78981c47abadc357c2b8b66844/users/c6405a9c349a4254aea3023dd5880169/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.732801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ce503-6a8a-46f9-a49d-9cb9112b85f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14936c78981c47abadc357c2b8b66844, user_id=c6405a9c349a4254aea3023dd5880169, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:05.736418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ce503-6a8a-46f9-a49d-9cb9112b85f5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:05.750728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1057/2123] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:05 2022] PUT /identity/v3/projects/14936c78981c47abadc357c2b8b66844/users/c6405a9c349a4254aea3023dd5880169/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:05.760515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45adddb-5c60-41f9-b1d9-2c8bed6db711 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.760811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45adddb-5c60-41f9-b1d9-2c8bed6db711 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.761086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45adddb-5c60-41f9-b1d9-2c8bed6db711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.781458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4021cfdf4e0040e387314c4ca0bc62f6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:05.791817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4021cfdf4e0040e387314c4ca0bc62f6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:05.793794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c45adddb-5c60-41f9-b1d9-2c8bed6db711 None None] MFA Rules not processed for user `c6405a9c349a4254aea3023dd5880169`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:05.795718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c45adddb-5c60-41f9-b1d9-2c8bed6db711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:05.805515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4021cfdf4e0040e387314c4ca0bc62f6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:05.806428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4021cfdf4e0040e387314c4ca0bc62f6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:05.807072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-20f148fe-8dda-41fa-b9fa-f2dd92ef7f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4021cfdf4e0040e387314c4ca0bc62f6'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:05.820805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1067/2124] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:05 2022] DELETE /identity/v3/users/4021cfdf4e0040e387314c4ca0bc62f6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:05.826065 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dab2ca-6a95-490f-ae4e-958c1a7b89c8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.826321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dab2ca-6a95-490f-ae4e-958c1a7b89c8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.826652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dab2ca-6a95-490f-ae4e-958c1a7b89c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:05.850003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-79dab2ca-6a95-490f-ae4e-958c1a7b89c8 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:05.851231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-79dab2ca-6a95-490f-ae4e-958c1a7b89c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:05.855508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:05.886643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1058/2125] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:05.898275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45adddb-5c60-41f9-b1d9-2c8bed6db711 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.899737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c45adddb-5c60-41f9-b1d9-2c8bed6db711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:05.916908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1068/2126] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:05.926538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dab2ca-6a95-490f-ae4e-958c1a7b89c8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:05.927912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-79dab2ca-6a95-490f-ae4e-958c1a7b89c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:05.940935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:05.958017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45adddb-5c60-41f9-b1d9-2c8bed6db711 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.985891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dab2ca-6a95-490f-ae4e-958c1a7b89c8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:05.989497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac9a6a-45de-4eb3-9b73-228463369d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:05.990213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac9a6a-45de-4eb3-9b73-228463369d4d service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:05.990498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac9a6a-45de-4eb3-9b73-228463369d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:05.990770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac9a6a-45de-4eb3-9b73-228463369d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.000158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac9a6a-45de-4eb3-9b73-228463369d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:06.002707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac9a6a-45de-4eb3-9b73-228463369d4d service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:06.008941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378f56-6fc5-4b92-a55f-da4c88e56599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:06.009548 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378f56-6fc5-4b92-a55f-da4c88e56599 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.009822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378f56-6fc5-4b92-a55f-da4c88e56599 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.010093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378f56-6fc5-4b92-a55f-da4c88e56599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.012043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-29378f56-6fc5-4b92-a55f-da4c88e56599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:06.013542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1069/2127] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:04:06.083823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378f56-6fc5-4b92-a55f-da4c88e56599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:06.085890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378f56-6fc5-4b92-a55f-da4c88e56599 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:06.097979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1059/2128] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:04:06.315106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac9a6a-45de-4eb3-9b73-228463369d4d service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:06.316570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2aac9a6a-45de-4eb3-9b73-228463369d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:06.334442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac9a6a-45de-4eb3-9b73-228463369d4d service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:06.340773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:06.341332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.341732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.342430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] PATH_INFO: `/v3/projects/d55d42ab558d46758144adae5e0b4193` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.343686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55d42ab558d46758144adae5e0b4193)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:06.345514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:06.405372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55d42ab558d46758144adae5e0b4193', 'request_id': 'req-38d02622-7a35-4b44-a114-7c0f9deb1ec8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:06.414326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55d42ab558d46758144adae5e0b4193', 'request_id': 'req-38d02622-7a35-4b44-a114-7c0f9deb1ec8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:06.423925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55d42ab558d46758144adae5e0b4193', 'request_id': 'req-38d02622-7a35-4b44-a114-7c0f9deb1ec8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:06.443121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] The token cache is being invalidate because project d55d42ab558d46758144adae5e0b4193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:06.443121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:06.443121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-38d02622-7a35-4b44-a114-7c0f9deb1ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:06.443121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1070/2129] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:06 2022] DELETE /identity/v3/projects/d55d42ab558d46758144adae5e0b4193 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:06.447955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20daa-c0b5-4800-ba45-17ddff3d5bf4 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.448243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20daa-c0b5-4800-ba45-17ddff3d5bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.448547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20daa-c0b5-4800-ba45-17ddff3d5bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.468247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-54d20daa-c0b5-4800-ba45-17ddff3d5bf4 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:06.469416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-54d20daa-c0b5-4800-ba45-17ddff3d5bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:06.539336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1060/2130] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:06.547283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3bf8e-a65d-498d-a3e9-e73151105a28 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.547565 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3bf8e-a65d-498d-a3e9-e73151105a28 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.547837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3bf8e-a65d-498d-a3e9-e73151105a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.568702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-88f3bf8e-a65d-498d-a3e9-e73151105a28 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:06.569901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-88f3bf8e-a65d-498d-a3e9-e73151105a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:06.578862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1071/2131] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:06.583198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d20daa-c0b5-4800-ba45-17ddff3d5bf4 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:06.592742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d20daa-c0b5-4800-ba45-17ddff3d5bf4 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:06.597354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08d91d-314c-4956-8b75-6940725b219a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:06.597955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08d91d-314c-4956-8b75-6940725b219a admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.598281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08d91d-314c-4956-8b75-6940725b219a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.598555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08d91d-314c-4956-8b75-6940725b219a admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.599424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08d91d-314c-4956-8b75-6940725b219a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:04:06.599719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08d91d-314c-4956-8b75-6940725b219a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:06.601309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08d91d-314c-4956-8b75-6940725b219a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:06.614988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1061/2132] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:04:06.619125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3bf8e-a65d-498d-a3e9-e73151105a28 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:06.629337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3bf8e-a65d-498d-a3e9-e73151105a28 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:06.634142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca9756-55e2-4fce-9ee4-8bde168b06f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:06.634740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca9756-55e2-4fce-9ee4-8bde168b06f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.635045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca9756-55e2-4fce-9ee4-8bde168b06f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.635315 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca9756-55e2-4fce-9ee4-8bde168b06f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.635953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca9756-55e2-4fce-9ee4-8bde168b06f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:06.637924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca9756-55e2-4fce-9ee4-8bde168b06f8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:06.669114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1072/2133] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:06 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:06.673957 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08d91d-314c-4956-8b75-6940725b219a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:06.688593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08d91d-314c-4956-8b75-6940725b219a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:06.721979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:06.751529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d20d4-a779-43af-b5d1-f5f9867bc3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:06.752282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d20d4-a779-43af-b5d1-f5f9867bc3dd admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.752437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d20d4-a779-43af-b5d1-f5f9867bc3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.752706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d20d4-a779-43af-b5d1-f5f9867bc3dd admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.753325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d20d4-a779-43af-b5d1-f5f9867bc3dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:06.755291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d20d4-a779-43af-b5d1-f5f9867bc3dd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:06.777487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1062/2134] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:06 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:06.782166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca9756-55e2-4fce-9ee4-8bde168b06f8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:06.792385 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca9756-55e2-4fce-9ee4-8bde168b06f8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:06.797467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407d0cb-7bbb-4c79-a0c5-67dbd45315f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:06.798056 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d0cb-7bbb-4c79-a0c5-67dbd45315f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.798334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d0cb-7bbb-4c79-a0c5-67dbd45315f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.798604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d0cb-7bbb-4c79-a0c5-67dbd45315f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.799274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407d0cb-7bbb-4c79-a0c5-67dbd45315f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:06.800863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407d0cb-7bbb-4c79-a0c5-67dbd45315f9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:06.802048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:06.812633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1073/2135] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:06.816407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d20d4-a779-43af-b5d1-f5f9867bc3dd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:06.826032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d20d4-a779-43af-b5d1-f5f9867bc3dd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:06.830473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16672923-f3b2-4c2b-9a7b-afc0ce3b00f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:06.831077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16672923-f3b2-4c2b-9a7b-afc0ce3b00f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.831354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16672923-f3b2-4c2b-9a7b-afc0ce3b00f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.831621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16672923-f3b2-4c2b-9a7b-afc0ce3b00f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.832257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16672923-f3b2-4c2b-9a7b-afc0ce3b00f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:06.833819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16672923-f3b2-4c2b-9a7b-afc0ce3b00f6 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:06.843972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1063/2136] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:06.847896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407d0cb-7bbb-4c79-a0c5-67dbd45315f9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:06.858152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407d0cb-7bbb-4c79-a0c5-67dbd45315f9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:06.862682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880e397-3651-4a9f-9727-c300fe0a13ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:06.863380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880e397-3651-4a9f-9727-c300fe0a13ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.863660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880e397-3651-4a9f-9727-c300fe0a13ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.863934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880e397-3651-4a9f-9727-c300fe0a13ca admin admin] PATH_INFO: `/v3/projects/0c42b50ff98c40a6b2f47f289538a330/users/55ae4dbafed74e9dbeab9538898cf1f6/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.878873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880e397-3651-4a9f-9727-c300fe0a13ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c42b50ff98c40a6b2f47f289538a330, user_id=55ae4dbafed74e9dbeab9538898cf1f6, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:06.881583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880e397-3651-4a9f-9727-c300fe0a13ca admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:06.895332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1074/2137] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:06 2022] PUT /identity/v3/projects/0c42b50ff98c40a6b2f47f289538a330/users/55ae4dbafed74e9dbeab9538898cf1f6/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:06.901684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d409df-cac2-4387-813f-72fd446ca402 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:06.901967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d409df-cac2-4387-813f-72fd446ca402 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:06.902232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d409df-cac2-4387-813f-72fd446ca402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:06.922945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e2d409df-cac2-4387-813f-72fd446ca402 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:06.924317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e2d409df-cac2-4387-813f-72fd446ca402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:06.983560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1064/2138] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:06.993093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880e397-3651-4a9f-9727-c300fe0a13ca admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:06.994798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b880e397-3651-4a9f-9727-c300fe0a13ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:07.050425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880e397-3651-4a9f-9727-c300fe0a13ca admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:07.053928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53030e09-8a4b-449e-974e-a9c6d62ac107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:07.054521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53030e09-8a4b-449e-974e-a9c6d62ac107 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:07.054795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53030e09-8a4b-449e-974e-a9c6d62ac107 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:07.055105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53030e09-8a4b-449e-974e-a9c6d62ac107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:07.063837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53030e09-8a4b-449e-974e-a9c6d62ac107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:07.065233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53030e09-8a4b-449e-974e-a9c6d62ac107 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:07.074291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1075/2139] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:07.684991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35a6e4-b717-4e2b-a89b-23227388edf6 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:07.685471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35a6e4-b717-4e2b-a89b-23227388edf6 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:07.685592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35a6e4-b717-4e2b-a89b-23227388edf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:07.718291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-9d35a6e4-b717-4e2b-a89b-23227388edf6 None None] MFA Rules not processed for user `55ae4dbafed74e9dbeab9538898cf1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:07.719580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9d35a6e4-b717-4e2b-a89b-23227388edf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:07.813303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1065/2140] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:07.828442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53030e09-8a4b-449e-974e-a9c6d62ac107 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:07.830459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-53030e09-8a4b-449e-974e-a9c6d62ac107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:07.904414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53030e09-8a4b-449e-974e-a9c6d62ac107 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:07.908285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4913dd2-3a17-4829-a106-b1b17762b58f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:07.908919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4913dd2-3a17-4829-a106-b1b17762b58f service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:07.909195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4913dd2-3a17-4829-a106-b1b17762b58f service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:07.909466 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4913dd2-3a17-4829-a106-b1b17762b58f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:07.918815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4913dd2-3a17-4829-a106-b1b17762b58f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:07.920284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4913dd2-3a17-4829-a106-b1b17762b58f service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:07.930022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1076/2141] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:09.301868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df109c9c-31a8-4582-aec5-02fd6398908f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:09.302618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df109c9c-31a8-4582-aec5-02fd6398908f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:09.302618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df109c9c-31a8-4582-aec5-02fd6398908f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:09.323103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-df109c9c-31a8-4582-aec5-02fd6398908f None None] MFA Rules not processed for user `55ae4dbafed74e9dbeab9538898cf1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:09.324290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-df109c9c-31a8-4582-aec5-02fd6398908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:09.331438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1066/2142] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:09.742612 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4913dd2-3a17-4829-a106-b1b17762b58f service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:09.744052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c4913dd2-3a17-4829-a106-b1b17762b58f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:09.758416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4913dd2-3a17-4829-a106-b1b17762b58f service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:09.764116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:09.764809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:09.765156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:09.765498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] PATH_INFO: `/v3/users/6b7b3cb86eab456fa239bc61fe7ba289` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:09.766679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b7b3cb86eab456fa239bc61fe7ba289)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:09.768694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:09.826842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b7b3cb86eab456fa239bc61fe7ba289'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:09.835605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b7b3cb86eab456fa239bc61fe7ba289'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:09.859947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:09.861615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b7b3cb86eab456fa239bc61fe7ba289'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:09.875792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b7b3cb86eab456fa239bc61fe7ba289'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:09.879996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b7b3cb86eab456fa239bc61fe7ba289'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:09.901819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1077/2143] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:09 2022] DELETE /identity/v3/users/6b7b3cb86eab456fa239bc61fe7ba289 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:09.906529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a14c0-e460-4819-936b-8fa0a97aac5f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:09.906828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a14c0-e460-4819-936b-8fa0a97aac5f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:09.907118 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a14c0-e460-4819-936b-8fa0a97aac5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:09.930118 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ce1a14c0-e460-4819-936b-8fa0a97aac5f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:09.931882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ce1a14c0-e460-4819-936b-8fa0a97aac5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:09.978533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:10.013026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1067/2144] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:10.024053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:10.025350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:10.095916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a2340-6275-40fd-b7f5-20a99fae10db admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:10.100691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c947d5b-81d4-4c76-a9c7-af1222a2b745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:10.101316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c947d5b-81d4-4c76-a9c7-af1222a2b745 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:10.101591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c947d5b-81d4-4c76-a9c7-af1222a2b745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:10.102179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c947d5b-81d4-4c76-a9c7-af1222a2b745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:10.116006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c947d5b-81d4-4c76-a9c7-af1222a2b745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:10.117392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c947d5b-81d4-4c76-a9c7-af1222a2b745 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:10.129268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1078/2145] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:10.199576 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a14c0-e460-4819-936b-8fa0a97aac5f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:10.201156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ce1a14c0-e460-4819-936b-8fa0a97aac5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:10.224052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a14c0-e460-4819-936b-8fa0a97aac5f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:10.238250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:10.239098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:10.239486 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:10.239966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] PATH_INFO: `/v3/users/6340a2593f6b4901bc8c4b954e32476e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:10.244410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6340a2593f6b4901bc8c4b954e32476e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:10.249116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:10.379350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6340a2593f6b4901bc8c4b954e32476e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.414841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6340a2593f6b4901bc8c4b954e32476e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.433495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6340a2593f6b4901bc8c4b954e32476e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.453734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6340a2593f6b4901bc8c4b954e32476e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.454685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6340a2593f6b4901bc8c4b954e32476e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.461599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1068/2146] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:10 2022] DELETE /identity/v3/users/6340a2593f6b4901bc8c4b954e32476e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:10.467432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad86680-b87b-4787-8874-dbce93a13208 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:10.467703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad86680-b87b-4787-8874-dbce93a13208 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:10.468006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad86680-b87b-4787-8874-dbce93a13208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:10.536001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8ad86680-b87b-4787-8874-dbce93a13208 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:10.541454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8ad86680-b87b-4787-8874-dbce93a13208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:10.659856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1079/2147] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:10.681137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:10.683255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:10.745791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad86680-b87b-4787-8874-dbce93a13208 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:10.748091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8ad86680-b87b-4787-8874-dbce93a13208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:10.762406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad86680-b87b-4787-8874-dbce93a13208 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:10.768327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:10.769165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:10.769592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:10.770009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] PATH_INFO: `/v3/projects/25acd9dcf66d40078dd7b6614d8803e1` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:10.771519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=25acd9dcf66d40078dd7b6614d8803e1)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:10.774436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:10.851475 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f307d4-5018-4d52-b506-1717bbde9539 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:10.870567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25acd9dcf66d40078dd7b6614d8803e1', 'request_id': 'req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.881850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25acd9dcf66d40078dd7b6614d8803e1', 'request_id': 'req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.893849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25acd9dcf66d40078dd7b6614d8803e1', 'request_id': 'req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.922710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] The token cache is being invalidate because project 25acd9dcf66d40078dd7b6614d8803e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:10.923058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.923726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:10.923828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:10.924449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:10.924726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:10.924995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:10.925279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1080/2148] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:10 2022] DELETE /identity/v3/projects/25acd9dcf66d40078dd7b6614d8803e1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:10.927378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:10.930035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:10.931730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:10.951024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:10.951864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:11.010434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:11.011927 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:11.013857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d9507-fc2a-4c3e-915c-eb985fe908ad admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:11.020303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:11.020929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.021574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.021879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] PATH_INFO: `/v3/users/c6405a9c349a4254aea3023dd5880169` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.023126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6405a9c349a4254aea3023dd5880169)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:11.025078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1069/2149] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:11.025251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:11.092634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6405a9c349a4254aea3023dd5880169'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.101258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6405a9c349a4254aea3023dd5880169'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.114519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6405a9c349a4254aea3023dd5880169'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.115288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6405a9c349a4254aea3023dd5880169'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.115900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6405a9c349a4254aea3023dd5880169'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.130076 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1081/2150] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:10 2022] DELETE /identity/v3/users/c6405a9c349a4254aea3023dd5880169 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:11.320929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:11.322346 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:11.336837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e64b1-db88-432d-8b39-11a45584cc90 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:11.421852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:11.422872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.422872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.423158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] PATH_INFO: `/v3/projects/cf1721023e1545028ac04c9620fecaae` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.424627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf1721023e1545028ac04c9620fecaae)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:11.426617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:11.432231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:11.433753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:11.447955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ca40-59b4-4da2-9ea0-695cad9fe14a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:11.453425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:11.454039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.454312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.454587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] PATH_INFO: `/v3/projects/14936c78981c47abadc357c2b8b66844` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.455760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=14936c78981c47abadc357c2b8b66844)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:11.457665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:11.488632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf1721023e1545028ac04c9620fecaae', 'request_id': 'req-576761be-423c-4029-9ea1-419635f93c3c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.497629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf1721023e1545028ac04c9620fecaae', 'request_id': 'req-576761be-423c-4029-9ea1-419635f93c3c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.507917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf1721023e1545028ac04c9620fecaae', 'request_id': 'req-576761be-423c-4029-9ea1-419635f93c3c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.514621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14936c78981c47abadc357c2b8b66844', 'request_id': 'req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.519180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] The token cache is being invalidate because project cf1721023e1545028ac04c9620fecaae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:11.519474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.520093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-576761be-423c-4029-9ea1-419635f93c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.525632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1070/2151] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:11 2022] DELETE /identity/v3/projects/cf1721023e1545028ac04c9620fecaae => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:11.525791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14936c78981c47abadc357c2b8b66844', 'request_id': 'req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.527033 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed75ca9-bd33-46d9-ac68-3489580b224c None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.527269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed75ca9-bd33-46d9-ac68-3489580b224c None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.527546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed75ca9-bd33-46d9-ac68-3489580b224c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.535015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14936c78981c47abadc357c2b8b66844', 'request_id': 'req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.548327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-4ed75ca9-bd33-46d9-ac68-3489580b224c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:11.549596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4ed75ca9-bd33-46d9-ac68-3489580b224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:11.576701 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] The token cache is being invalidate because project 14936c78981c47abadc357c2b8b66844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:11.577533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.579506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bc685d3-13f5-45aa-9c6d-c0875937d8ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:11.581674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1082/2152] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:11 2022] DELETE /identity/v3/projects/14936c78981c47abadc357c2b8b66844 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:11.587151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07ac5a-8445-4ff1-8322-17efb4f96aad None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.587582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07ac5a-8445-4ff1-8322-17efb4f96aad None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.587952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07ac5a-8445-4ff1-8322-17efb4f96aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.613156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:11.627286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cc07ac5a-8445-4ff1-8322-17efb4f96aad None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:11.630039 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cc07ac5a-8445-4ff1-8322-17efb4f96aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:11.635045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1071/2153] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:11.644528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a99faf-cf10-41eb-b8aa-452287b943dc None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.645028 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a99faf-cf10-41eb-b8aa-452287b943dc None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.645295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a99faf-cf10-41eb-b8aa-452287b943dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.655947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1083/2154] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:11.666274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592228-185b-40cc-b5e4-c08a671eedb7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.667202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592228-185b-40cc-b5e4-c08a671eedb7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.667644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592228-185b-40cc-b5e4-c08a671eedb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.681354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-20a99faf-cf10-41eb-b8aa-452287b943dc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:11.683048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-20a99faf-cf10-41eb-b8aa-452287b943dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:11.695075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a9592228-185b-40cc-b5e4-c08a671eedb7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:11.697487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1072/2155] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:11.698703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a99faf-cf10-41eb-b8aa-452287b943dc None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:11.699356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a9592228-185b-40cc-b5e4-c08a671eedb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:11.716743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a99faf-cf10-41eb-b8aa-452287b943dc None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:11.718922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1084/2156] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:11.719951 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9592228-185b-40cc-b5e4-c08a671eedb7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:11.728584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:11.734630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9592228-185b-40cc-b5e4-c08a671eedb7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:11.739270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf91f5-3f13-442a-b39a-3bdec811c99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:11.742494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf91f5-3f13-442a-b39a-3bdec811c99b admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.742972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf91f5-3f13-442a-b39a-3bdec811c99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.743406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf91f5-3f13-442a-b39a-3bdec811c99b admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.743989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aaf0e-93d3-40b7-83d6-ee54e759ef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:11.744635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf91f5-3f13-442a-b39a-3bdec811c99b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:04:11.744904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aaf0e-93d3-40b7-83d6-ee54e759ef01 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.745376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aaf0e-93d3-40b7-83d6-ee54e759ef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.745724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aaf0e-93d3-40b7-83d6-ee54e759ef01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.746488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95aaf0e-93d3-40b7-83d6-ee54e759ef01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:04:11.747101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95aaf0e-93d3-40b7-83d6-ee54e759ef01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:11.747322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf91f5-3f13-442a-b39a-3bdec811c99b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:11.748864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95aaf0e-93d3-40b7-83d6-ee54e759ef01 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:11.749963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf91f5-3f13-442a-b39a-3bdec811c99b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:11.773219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1085/2157] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:04:11.774400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aaf0e-93d3-40b7-83d6-ee54e759ef01 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:11.797892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aaf0e-93d3-40b7-83d6-ee54e759ef01 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:11.801980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1073/2158] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:04:11.804112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf91f5-3f13-442a-b39a-3bdec811c99b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:11.813233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad396bfd-aa61-440c-96c9-13d2f87646f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:11.814298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad396bfd-aa61-440c-96c9-13d2f87646f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.815382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad396bfd-aa61-440c-96c9-13d2f87646f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.815796 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad396bfd-aa61-440c-96c9-13d2f87646f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.817582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad396bfd-aa61-440c-96c9-13d2f87646f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:11.821636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad396bfd-aa61-440c-96c9-13d2f87646f9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:11.822069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf91f5-3f13-442a-b39a-3bdec811c99b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:11.827854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db78438-89e0-4a5c-b0ed-010d55820beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:11.828498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db78438-89e0-4a5c-b0ed-010d55820beb admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:11.828782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db78438-89e0-4a5c-b0ed-010d55820beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:11.829637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db78438-89e0-4a5c-b0ed-010d55820beb admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:11.829932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db78438-89e0-4a5c-b0ed-010d55820beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:11.832242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db78438-89e0-4a5c-b0ed-010d55820beb admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:11.871125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1074/2159] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:11 2022] POST /identity/v3/projects => generated 397 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:11.874529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db78438-89e0-4a5c-b0ed-010d55820beb admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:11.884392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1086/2160] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:11 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:11.888673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad396bfd-aa61-440c-96c9-13d2f87646f9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.003728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db78438-89e0-4a5c-b0ed-010d55820beb admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.018935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad396bfd-aa61-440c-96c9-13d2f87646f9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.027000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b655-83f7-4c33-b6ca-aaab895f8a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.027869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b655-83f7-4c33-b6ca-aaab895f8a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.029333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b655-83f7-4c33-b6ca-aaab895f8a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.029449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b655-83f7-4c33-b6ca-aaab895f8a91 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.030439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3b655-83f7-4c33-b6ca-aaab895f8a91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.032376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3b655-83f7-4c33-b6ca-aaab895f8a91 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.077064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1087/2161] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:11 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:12.080822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b655-83f7-4c33-b6ca-aaab895f8a91 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.106702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b655-83f7-4c33-b6ca-aaab895f8a91 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.119716 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a03b5-a001-4881-8898-113abc57025b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.120333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a03b5-a001-4881-8898-113abc57025b admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.120621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a03b5-a001-4881-8898-113abc57025b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.120906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a03b5-a001-4881-8898-113abc57025b admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.123123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a03b5-a001-4881-8898-113abc57025b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.131710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a03b5-a001-4881-8898-113abc57025b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.141661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72049b-c8ce-4c2b-9b8e-9af6c711c03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.142484 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72049b-c8ce-4c2b-9b8e-9af6c711c03f admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.142862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72049b-c8ce-4c2b-9b8e-9af6c711c03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.143289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72049b-c8ce-4c2b-9b8e-9af6c711c03f admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.144119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72049b-c8ce-4c2b-9b8e-9af6c711c03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.146653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72049b-c8ce-4c2b-9b8e-9af6c711c03f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.169805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1088/2162] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:12.172892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a03b5-a001-4881-8898-113abc57025b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.189679 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1075/2163] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:11 2022] POST /identity/v3/users => generated 299 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:12.194313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72049b-c8ce-4c2b-9b8e-9af6c711c03f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.199401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a03b5-a001-4881-8898-113abc57025b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.207668 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72049b-c8ce-4c2b-9b8e-9af6c711c03f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.212044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe5834-9ea6-4440-b7bf-3b179b6dbff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.213451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe5834-9ea6-4440-b7bf-3b179b6dbff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.213898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652691ea-0670-4554-bd95-b93b9fe950df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.214031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe5834-9ea6-4440-b7bf-3b179b6dbff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.214479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe5834-9ea6-4440-b7bf-3b179b6dbff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.214816 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652691ea-0670-4554-bd95-b93b9fe950df admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.215233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652691ea-0670-4554-bd95-b93b9fe950df admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.216867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652691ea-0670-4554-bd95-b93b9fe950df admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.216992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652691ea-0670-4554-bd95-b93b9fe950df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.217222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe5834-9ea6-4440-b7bf-3b179b6dbff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.218983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652691ea-0670-4554-bd95-b93b9fe950df admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.219900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe5834-9ea6-4440-b7bf-3b179b6dbff9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.235452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1076/2164] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:12.238799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1089/2165] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:12.242767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe5834-9ea6-4440-b7bf-3b179b6dbff9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.243077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652691ea-0670-4554-bd95-b93b9fe950df admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.257936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe5834-9ea6-4440-b7bf-3b179b6dbff9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.260393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652691ea-0670-4554-bd95-b93b9fe950df admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.264731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f5b3c-145c-4abb-824c-01cd7222eecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.265649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5b3c-145c-4abb-824c-01cd7222eecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.265996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9567ce5-9d6e-4797-a4dc-364a2b020405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.266299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5b3c-145c-4abb-824c-01cd7222eecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.266726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5b3c-145c-4abb-824c-01cd7222eecf admin admin] PATH_INFO: `/v3/projects/8a1f9a2e54fb44c1a5ad56cb1dd6b8f2/users/ed687d192fbf4bde9b5c0349b1ef7ac8/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.267852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9567ce5-9d6e-4797-a4dc-364a2b020405 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.268890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9567ce5-9d6e-4797-a4dc-364a2b020405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.269272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9567ce5-9d6e-4797-a4dc-364a2b020405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.271126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9567ce5-9d6e-4797-a4dc-364a2b020405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.274704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9567ce5-9d6e-4797-a4dc-364a2b020405 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.296652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1077/2166] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:12.300725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f5b3c-145c-4abb-824c-01cd7222eecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1f9a2e54fb44c1a5ad56cb1dd6b8f2, user_id=ed687d192fbf4bde9b5c0349b1ef7ac8, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.305233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f5b3c-145c-4abb-824c-01cd7222eecf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.324234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9567ce5-9d6e-4797-a4dc-364a2b020405 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.337365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9567ce5-9d6e-4797-a4dc-364a2b020405 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.358677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:12.359459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1090/2167] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:12 2022] PUT /identity/v3/projects/8a1f9a2e54fb44c1a5ad56cb1dd6b8f2/users/ed687d192fbf4bde9b5c0349b1ef7ac8/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:12.376999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a771a-cb2a-4ef5-be0b-e2ae95007bf1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.377292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a771a-cb2a-4ef5-be0b-e2ae95007bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.377572 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a771a-cb2a-4ef5-be0b-e2ae95007bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.443787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-795a771a-cb2a-4ef5-be0b-e2ae95007bf1 None None] MFA Rules not processed for user `ed687d192fbf4bde9b5c0349b1ef7ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:12.445334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-795a771a-cb2a-4ef5-be0b-e2ae95007bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:12.448745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e64808-fd75-4d93-86e4-4ff8d054ce69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.449483 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e64808-fd75-4d93-86e4-4ff8d054ce69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.449770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e64808-fd75-4d93-86e4-4ff8d054ce69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.450037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e64808-fd75-4d93-86e4-4ff8d054ce69 admin admin] PATH_INFO: `/v3/projects/187a2119d50c4d7f8cd7c420c2969595/users/4736989f3b544a91919a4852cabe7e20/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.484608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e64808-fd75-4d93-86e4-4ff8d054ce69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=187a2119d50c4d7f8cd7c420c2969595, user_id=4736989f3b544a91919a4852cabe7e20, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.487711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e64808-fd75-4d93-86e4-4ff8d054ce69 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.510332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1078/2168] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:12 2022] PUT /identity/v3/projects/187a2119d50c4d7f8cd7c420c2969595/users/4736989f3b544a91919a4852cabe7e20/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:12.520469 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6812f-93c0-451f-8d12-ef6eafca7f91 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.520755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6812f-93c0-451f-8d12-ef6eafca7f91 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.521032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6812f-93c0-451f-8d12-ef6eafca7f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.550708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-eab6812f-93c0-451f-8d12-ef6eafca7f91 None None] MFA Rules not processed for user `4736989f3b544a91919a4852cabe7e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:12.551913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-eab6812f-93c0-451f-8d12-ef6eafca7f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:12.559472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1091/2169] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:12.560507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a771a-cb2a-4ef5-be0b-e2ae95007bf1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.562638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-795a771a-cb2a-4ef5-be0b-e2ae95007bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:12.572957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:12.581143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a771a-cb2a-4ef5-be0b-e2ae95007bf1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.629587 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1079/2170] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:12.630980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6812f-93c0-451f-8d12-ef6eafca7f91 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.661514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.662358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.662734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.663981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] PATH_INFO: `/v3/users/3267b05f29b840c5bd42d446117d8b16` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.665920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3267b05f29b840c5bd42d446117d8b16)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.668033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.709344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6812f-93c0-451f-8d12-ef6eafca7f91 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.715997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5a7b0-57a7-45e4-81d4-ec1fec113335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.717182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5a7b0-57a7-45e4-81d4-ec1fec113335 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.717822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5a7b0-57a7-45e4-81d4-ec1fec113335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.719206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5a7b0-57a7-45e4-81d4-ec1fec113335 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.719931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5a7b0-57a7-45e4-81d4-ec1fec113335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.721889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5a7b0-57a7-45e4-81d4-ec1fec113335 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.729309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3267b05f29b840c5bd42d446117d8b16'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:12.737052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3267b05f29b840c5bd42d446117d8b16'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:12.749938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3267b05f29b840c5bd42d446117d8b16'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:12.750907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3267b05f29b840c5bd42d446117d8b16'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:12.751579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3267b05f29b840c5bd42d446117d8b16'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:12.754213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1080/2171] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:12 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:12.757547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5a7b0-57a7-45e4-81d4-ec1fec113335 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.759515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ebb5a7b0-57a7-45e4-81d4-ec1fec113335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:12.768213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1092/2172] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:12 2022] DELETE /identity/v3/users/3267b05f29b840c5bd42d446117d8b16 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:12.769230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.770643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:12.786121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:12.825814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5a7b0-57a7-45e4-81d4-ec1fec113335 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.842313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d35c17-3e8d-4133-b0ed-c223c54596d9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.847147 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a98d53-d2d6-49f7-9e0a-c1a6d4695f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.847759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a98d53-d2d6-49f7-9e0a-c1a6d4695f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.848037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a98d53-d2d6-49f7-9e0a-c1a6d4695f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.848310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a98d53-d2d6-49f7-9e0a-c1a6d4695f53 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.848927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a98d53-d2d6-49f7-9e0a-c1a6d4695f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.850839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a98d53-d2d6-49f7-9e0a-c1a6d4695f53 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.874511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1093/2173] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:12 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:12.876505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31c94a-f1f0-4b43-a4c5-07ade372efb7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.877419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31c94a-f1f0-4b43-a4c5-07ade372efb7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.877809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31c94a-f1f0-4b43-a4c5-07ade372efb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.885008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3e06b-2b73-4d7e-b618-e197eb90970f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.885621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3e06b-2b73-4d7e-b618-e197eb90970f service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.885900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3e06b-2b73-4d7e-b618-e197eb90970f service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.886173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3e06b-2b73-4d7e-b618-e197eb90970f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.888672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7ae3e06b-2b73-4d7e-b618-e197eb90970f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:12.898068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bb31c94a-f1f0-4b43-a4c5-07ade372efb7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:12.899433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bb31c94a-f1f0-4b43-a4c5-07ade372efb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:12.908159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1094/2174] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:12.908984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31c94a-f1f0-4b43-a4c5-07ade372efb7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.915936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:12.923468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31c94a-f1f0-4b43-a4c5-07ade372efb7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:12.929983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76b643-63f0-4434-b422-2a9afdec8750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:12.931233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76b643-63f0-4434-b422-2a9afdec8750 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:12.931233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76b643-63f0-4434-b422-2a9afdec8750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:12.931233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76b643-63f0-4434-b422-2a9afdec8750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:12.932186 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76b643-63f0-4434-b422-2a9afdec8750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.934082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76b643-63f0-4434-b422-2a9afdec8750 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.945016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1095/2175] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:12.945683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76b643-63f0-4434-b422-2a9afdec8750 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.947406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0e76b643-63f0-4434-b422-2a9afdec8750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:12.967235 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3e06b-2b73-4d7e-b618-e197eb90970f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:12.969099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3e06b-2b73-4d7e-b618-e197eb90970f service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:12.981973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1081/2176] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:12 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:12.983554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3e06b-2b73-4d7e-b618-e197eb90970f service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:12.994778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3e06b-2b73-4d7e-b618-e197eb90970f service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:13.005899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92447384-3d4b-4bb2-a92e-01ea5cb584c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:13.006745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92447384-3d4b-4bb2-a92e-01ea5cb584c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:13.007198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92447384-3d4b-4bb2-a92e-01ea5cb584c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:13.007607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92447384-3d4b-4bb2-a92e-01ea5cb584c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:13.009414 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92447384-3d4b-4bb2-a92e-01ea5cb584c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:13.011398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92447384-3d4b-4bb2-a92e-01ea5cb584c1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:13.013995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76b643-63f0-4434-b422-2a9afdec8750 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:13.022077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae93b4-95f2-485c-83b3-936ed37b5c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:13.022077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae93b4-95f2-485c-83b3-936ed37b5c74 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:13.022077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae93b4-95f2-485c-83b3-936ed37b5c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:13.022077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae93b4-95f2-485c-83b3-936ed37b5c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:13.033183 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae93b4-95f2-485c-83b3-936ed37b5c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:13.034621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae93b4-95f2-485c-83b3-936ed37b5c74 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:13.035109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1082/2177] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:13.039682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92447384-3d4b-4bb2-a92e-01ea5cb584c1 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:13.051420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1096/2178] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:13.070062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92447384-3d4b-4bb2-a92e-01ea5cb584c1 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:13.076276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23154f75-83bb-4d08-9e5a-a034751bd70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:13.076996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23154f75-83bb-4d08-9e5a-a034751bd70d admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:13.077277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23154f75-83bb-4d08-9e5a-a034751bd70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:13.077549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23154f75-83bb-4d08-9e5a-a034751bd70d admin admin] PATH_INFO: `/v3/projects/2db23f650dce4b7bb3562e100e37ade2/users/5ff6b544ab1447e29b99d81ffcab60e0/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:13.095924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23154f75-83bb-4d08-9e5a-a034751bd70d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db23f650dce4b7bb3562e100e37ade2, user_id=5ff6b544ab1447e29b99d81ffcab60e0, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:13.098774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23154f75-83bb-4d08-9e5a-a034751bd70d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:13.131065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1083/2179] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:13 2022] PUT /identity/v3/projects/2db23f650dce4b7bb3562e100e37ade2/users/5ff6b544ab1447e29b99d81ffcab60e0/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:13.131926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fac1f-72bb-431c-a8ca-9ec0b168df1c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:13.132177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fac1f-72bb-431c-a8ca-9ec0b168df1c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:13.132539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fac1f-72bb-431c-a8ca-9ec0b168df1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:13.190317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8c0fac1f-72bb-431c-a8ca-9ec0b168df1c None None] MFA Rules not processed for user `5ff6b544ab1447e29b99d81ffcab60e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:13.192784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8c0fac1f-72bb-431c-a8ca-9ec0b168df1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:13.299081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1097/2180] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:13.301998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23154f75-83bb-4d08-9e5a-a034751bd70d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:13.419478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:13.445294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fac1f-72bb-431c-a8ca-9ec0b168df1c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:13.447118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8c0fac1f-72bb-431c-a8ca-9ec0b168df1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:13.461901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23154f75-83bb-4d08-9e5a-a034751bd70d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:13.470269 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fac1f-72bb-431c-a8ca-9ec0b168df1c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:13.474492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93c394-3401-4544-8451-306c8a486fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:13.475137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93c394-3401-4544-8451-306c8a486fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:13.475421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93c394-3401-4544-8451-306c8a486fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:13.475695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93c394-3401-4544-8451-306c8a486fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:13.476387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93c394-3401-4544-8451-306c8a486fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:13.479443 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93c394-3401-4544-8451-306c8a486fdf admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:13.487336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:13.487336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:13.487806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:13.488400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] PATH_INFO: `/v3/projects/37d0c4f10bec4455828e2fbe3ac526ca` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:13.493543 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37d0c4f10bec4455828e2fbe3ac526ca)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:13.499515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:13.553262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1084/2181] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:13 2022] POST /identity/v3/projects => generated 383 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:13.558276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93c394-3401-4544-8451-306c8a486fdf admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:13.616337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37d0c4f10bec4455828e2fbe3ac526ca', 'request_id': 'req-cd2e0314-3872-43c4-b88c-60effc729aa7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:13.631628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37d0c4f10bec4455828e2fbe3ac526ca', 'request_id': 'req-cd2e0314-3872-43c4-b88c-60effc729aa7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:13.652587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37d0c4f10bec4455828e2fbe3ac526ca', 'request_id': 'req-cd2e0314-3872-43c4-b88c-60effc729aa7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:13.711693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] The token cache is being invalidate because project 37d0c4f10bec4455828e2fbe3ac526ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:13.711946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:13.712752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:13.716771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1098/2182] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:13 2022] DELETE /identity/v3/projects/37d0c4f10bec4455828e2fbe3ac526ca => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:13.721343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:13.723972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:13.757646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93c394-3401-4544-8451-306c8a486fdf admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:13.759006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:13.876215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e0314-3872-43c4-b88c-60effc729aa7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:13.877806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b4c09-858c-4864-9a86-8c9b484920d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:13.878593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b4c09-858c-4864-9a86-8c9b484920d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:13.879060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b4c09-858c-4864-9a86-8c9b484920d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:13.879490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b4c09-858c-4864-9a86-8c9b484920d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:13.880245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b4c09-858c-4864-9a86-8c9b484920d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:13.881788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:13.882325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b4c09-858c-4864-9a86-8c9b484920d5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:13.882604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:13.882969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:13.883449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] PATH_INFO: `/v3/users/df1b9e2b30694eb0a30e53f12906f016` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:13.884936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] RBAC: Authorizing `identity:delete_user(user_id=df1b9e2b30694eb0a30e53f12906f016)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:13.887091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:13.908055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1085/2183] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:13 2022] POST /identity/v3/users => generated 298 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:13.914771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b4c09-858c-4864-9a86-8c9b484920d5 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:13.917027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-050b4c09-858c-4864-9a86-8c9b484920d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:13.942315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b4c09-858c-4864-9a86-8c9b484920d5 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:13.949702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41007c84-60c4-42df-802f-767b8f01be27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:13.951441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41007c84-60c4-42df-802f-767b8f01be27 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:13.951850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41007c84-60c4-42df-802f-767b8f01be27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:13.952126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41007c84-60c4-42df-802f-767b8f01be27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:13.952907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41007c84-60c4-42df-802f-767b8f01be27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:13.955663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41007c84-60c4-42df-802f-767b8f01be27 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:13.974766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1086/2184] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:13.975764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41007c84-60c4-42df-802f-767b8f01be27 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:13.979768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df1b9e2b30694eb0a30e53f12906f016'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:13.990556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df1b9e2b30694eb0a30e53f12906f016'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:13.990791 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41007c84-60c4-42df-802f-767b8f01be27 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.008489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0cc0b-c3ba-4138-8184-e0529ef25a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.012698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0cc0b-c3ba-4138-8184-e0529ef25a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.013325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0cc0b-c3ba-4138-8184-e0529ef25a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.013608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0cc0b-c3ba-4138-8184-e0529ef25a2e admin admin] PATH_INFO: `/v3/projects/812bdaacfcb242d7a8f099d53695f2b5/users/05138ae913904cbda172304be4c7a9dd/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.029053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df1b9e2b30694eb0a30e53f12906f016'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:14.029883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df1b9e2b30694eb0a30e53f12906f016'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:14.030530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df1b9e2b30694eb0a30e53f12906f016'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:14.035524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0cc0b-c3ba-4138-8184-e0529ef25a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=812bdaacfcb242d7a8f099d53695f2b5, user_id=05138ae913904cbda172304be4c7a9dd, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.038535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0cc0b-c3ba-4138-8184-e0529ef25a2e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.045589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1099/2185] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:13 2022] DELETE /identity/v3/users/df1b9e2b30694eb0a30e53f12906f016 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:14.054122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1087/2186] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:13 2022] PUT /identity/v3/projects/812bdaacfcb242d7a8f099d53695f2b5/users/05138ae913904cbda172304be4c7a9dd/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:14.058120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:14.059647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:14.142352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b60b5-09cb-4a53-afcb-833513d0c77b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.176784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:14.205110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4301b-57d3-401d-bcd4-031ec907e90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.206074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4301b-57d3-401d-bcd4-031ec907e90b admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.206273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4301b-57d3-401d-bcd4-031ec907e90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.206574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4301b-57d3-401d-bcd4-031ec907e90b admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.207322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4301b-57d3-401d-bcd4-031ec907e90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.209297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4301b-57d3-401d-bcd4-031ec907e90b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.222923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1100/2187] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:14.224187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0cc0b-c3ba-4138-8184-e0529ef25a2e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:14.236825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0cc0b-c3ba-4138-8184-e0529ef25a2e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.241804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793984d-3d32-408a-85df-588e63323d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.242554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793984d-3d32-408a-85df-588e63323d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.242896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793984d-3d32-408a-85df-588e63323d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.243274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793984d-3d32-408a-85df-588e63323d08 admin admin] PATH_INFO: `/v3/projects/812bdaacfcb242d7a8f099d53695f2b5/users/05138ae913904cbda172304be4c7a9dd/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.245329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793984d-3d32-408a-85df-588e63323d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=812bdaacfcb242d7a8f099d53695f2b5, user_id=05138ae913904cbda172304be4c7a9dd, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.248734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793984d-3d32-408a-85df-588e63323d08 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.260839 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1088/2188] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:14 2022] PUT /identity/v3/projects/812bdaacfcb242d7a8f099d53695f2b5/users/05138ae913904cbda172304be4c7a9dd/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:14.271542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbfcae-ef18-4099-b177-83560750ff99 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.271888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbfcae-ef18-4099-b177-83560750ff99 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.272094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbfcae-ef18-4099-b177-83560750ff99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.300078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8ffbfcae-ef18-4099-b177-83560750ff99 None None] MFA Rules not processed for user `05138ae913904cbda172304be4c7a9dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:14.301270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8ffbfcae-ef18-4099-b177-83560750ff99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:14.355941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793984d-3d32-408a-85df-588e63323d08 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:14.357706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4793984d-3d32-408a-85df-588e63323d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:14.384104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1101/2189] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:14.391626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:14.394541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbfcae-ef18-4099-b177-83560750ff99 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:14.401533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8ffbfcae-ef18-4099-b177-83560750ff99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:14.436724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793984d-3d32-408a-85df-588e63323d08 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.441412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.442276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.442555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.444010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] PATH_INFO: `/v3/projects/27eaea7979df4282b78601f3ac0cd441` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.445186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=27eaea7979df4282b78601f3ac0cd441)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.447130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.458178 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbfcae-ef18-4099-b177-83560750ff99 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.461963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4f616-82c0-49de-91a6-869c303dba44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.462567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4f616-82c0-49de-91a6-869c303dba44 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.462847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4f616-82c0-49de-91a6-869c303dba44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.463170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4f616-82c0-49de-91a6-869c303dba44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.464915 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-26c4f616-82c0-49de-91a6-869c303dba44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:14.494594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27eaea7979df4282b78601f3ac0cd441', 'request_id': 'req-bded5881-27f4-465e-8400-ea8f4701c0aa'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:14.502876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27eaea7979df4282b78601f3ac0cd441', 'request_id': 'req-bded5881-27f4-465e-8400-ea8f4701c0aa'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:14.510902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27eaea7979df4282b78601f3ac0cd441', 'request_id': 'req-bded5881-27f4-465e-8400-ea8f4701c0aa'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:14.522237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] The token cache is being invalidate because project 27eaea7979df4282b78601f3ac0cd441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:14.522581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:14.523249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bded5881-27f4-465e-8400-ea8f4701c0aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:14.524743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1089/2190] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:14 2022] DELETE /identity/v3/projects/27eaea7979df4282b78601f3ac0cd441 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:14.524972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4f616-82c0-49de-91a6-869c303dba44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.526372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4f616-82c0-49de-91a6-869c303dba44 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.529872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d2eb3-1185-4c27-8e28-3bd14886beb1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.530160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d2eb3-1185-4c27-8e28-3bd14886beb1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.530435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d2eb3-1185-4c27-8e28-3bd14886beb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.536241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1102/2191] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:14.549922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-208d2eb3-1185-4c27-8e28-3bd14886beb1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:14.551250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-208d2eb3-1185-4c27-8e28-3bd14886beb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:14.612982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1090/2192] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:14.620505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed9074-9a1d-4744-8fb6-380493815c60 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.620792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed9074-9a1d-4744-8fb6-380493815c60 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.621062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed9074-9a1d-4744-8fb6-380493815c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.640598 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d3ed9074-9a1d-4744-8fb6-380493815c60 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:14.641767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d3ed9074-9a1d-4744-8fb6-380493815c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:14.649590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1103/2193] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:14.654567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d2eb3-1185-4c27-8e28-3bd14886beb1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:14.664714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d2eb3-1185-4c27-8e28-3bd14886beb1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.670022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01f48f-0d7d-4839-8016-773df2a75e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.670672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01f48f-0d7d-4839-8016-773df2a75e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.671126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01f48f-0d7d-4839-8016-773df2a75e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.671400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01f48f-0d7d-4839-8016-773df2a75e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.672148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01f48f-0d7d-4839-8016-773df2a75e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:04:14.672533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01f48f-0d7d-4839-8016-773df2a75e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.674345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01f48f-0d7d-4839-8016-773df2a75e0d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.686891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1091/2194] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:04:14.691443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed9074-9a1d-4744-8fb6-380493815c60 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:14.697992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:14.702251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed9074-9a1d-4744-8fb6-380493815c60 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.709810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa824e64-3306-4b86-91cf-e05e981b49f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.710505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa824e64-3306-4b86-91cf-e05e981b49f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.710772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa824e64-3306-4b86-91cf-e05e981b49f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.711062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa824e64-3306-4b86-91cf-e05e981b49f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.711704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa824e64-3306-4b86-91cf-e05e981b49f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.713803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa824e64-3306-4b86-91cf-e05e981b49f8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.746643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1104/2195] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:14 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:04:14.751161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01f48f-0d7d-4839-8016-773df2a75e0d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:14.761151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01f48f-0d7d-4839-8016-773df2a75e0d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.823089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:14.859442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830ed1c-5696-43b3-be86-38e142498708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.859442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830ed1c-5696-43b3-be86-38e142498708 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.859442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830ed1c-5696-43b3-be86-38e142498708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.859442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830ed1c-5696-43b3-be86-38e142498708 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.862440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830ed1c-5696-43b3-be86-38e142498708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.877818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830ed1c-5696-43b3-be86-38e142498708 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.905150 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1092/2196] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:14 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:04:14.908998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa824e64-3306-4b86-91cf-e05e981b49f8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:14.922094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa824e64-3306-4b86-91cf-e05e981b49f8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.929957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec37730-aea1-4328-91fe-73481753d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.930605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec37730-aea1-4328-91fe-73481753d2ce admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.930891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec37730-aea1-4328-91fe-73481753d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.931189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec37730-aea1-4328-91fe-73481753d2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.931855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec37730-aea1-4328-91fe-73481753d2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.934275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec37730-aea1-4328-91fe-73481753d2ce admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.945762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1105/2197] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:14.950165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830ed1c-5696-43b3-be86-38e142498708 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:14.963319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830ed1c-5696-43b3-be86-38e142498708 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:14.970167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6ae7d-e400-4f8c-a4a4-15e8f2daa05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:14.971585 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6ae7d-e400-4f8c-a4a4-15e8f2daa05f admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:14.972107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6ae7d-e400-4f8c-a4a4-15e8f2daa05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:14.972654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6ae7d-e400-4f8c-a4a4-15e8f2daa05f admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:14.973946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6ae7d-e400-4f8c-a4a4-15e8f2daa05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:14.976370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6ae7d-e400-4f8c-a4a4-15e8f2daa05f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:14.989126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1093/2198] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:14.993701 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec37730-aea1-4328-91fe-73481753d2ce admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:15.006805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec37730-aea1-4328-91fe-73481753d2ce admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:15.012908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2e3bf-86a9-459a-9cb4-1993daf2196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:15.013497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2e3bf-86a9-459a-9cb4-1993daf2196b admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.013770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2e3bf-86a9-459a-9cb4-1993daf2196b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.014095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2e3bf-86a9-459a-9cb4-1993daf2196b admin admin] PATH_INFO: `/v3/projects/ee7eab3a7c9a4405bbe490f0c9ca4a6c/users/53480141c7cf4aecb8c0653fd3cf30b5/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.032451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2e3bf-86a9-459a-9cb4-1993daf2196b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7eab3a7c9a4405bbe490f0c9ca4a6c, user_id=53480141c7cf4aecb8c0653fd3cf30b5, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:15.035277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2e3bf-86a9-459a-9cb4-1993daf2196b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:15.050421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1106/2199] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:14 2022] PUT /identity/v3/projects/ee7eab3a7c9a4405bbe490f0c9ca4a6c/users/53480141c7cf4aecb8c0653fd3cf30b5/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:15.062193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6f2e1-2e67-4c26-98c4-0467ca634650 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.062518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6f2e1-2e67-4c26-98c4-0467ca634650 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.062758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6f2e1-2e67-4c26-98c4-0467ca634650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.112864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-65a6f2e1-2e67-4c26-98c4-0467ca634650 None None] MFA Rules not processed for user `53480141c7cf4aecb8c0653fd3cf30b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:15.114826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-65a6f2e1-2e67-4c26-98c4-0467ca634650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:15.246378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1094/2200] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:15.251831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2e3bf-86a9-459a-9cb4-1993daf2196b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:15.337916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2e3bf-86a9-459a-9cb4-1993daf2196b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:15.343538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15a77-ac70-4f5c-84f2-8ec4b80a77f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:15.344326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15a77-ac70-4f5c-84f2-8ec4b80a77f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.344843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15a77-ac70-4f5c-84f2-8ec4b80a77f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.355094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15a77-ac70-4f5c-84f2-8ec4b80a77f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.356129 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b15a77-ac70-4f5c-84f2-8ec4b80a77f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:15.361904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b15a77-ac70-4f5c-84f2-8ec4b80a77f5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:15.408171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1107/2201] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:15 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:15.414123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6f2e1-2e67-4c26-98c4-0467ca634650 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:15.425722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6f2e1-2e67-4c26-98c4-0467ca634650 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:15.500825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f7493-3bfd-4674-8fc2-236f8b3c65ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:15.502157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f7493-3bfd-4674-8fc2-236f8b3c65ad admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.502969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f7493-3bfd-4674-8fc2-236f8b3c65ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.502969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f7493-3bfd-4674-8fc2-236f8b3c65ad admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.503347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f7493-3bfd-4674-8fc2-236f8b3c65ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:15.505530 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f7493-3bfd-4674-8fc2-236f8b3c65ad admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:15.526971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1095/2202] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:15 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:04:15.530935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15a77-ac70-4f5c-84f2-8ec4b80a77f5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:15.541173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15a77-ac70-4f5c-84f2-8ec4b80a77f5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:15.545789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77f589-30a3-46f5-817b-8d9db3c36892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:15.546377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77f589-30a3-46f5-817b-8d9db3c36892 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.546657 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77f589-30a3-46f5-817b-8d9db3c36892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.546930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77f589-30a3-46f5-817b-8d9db3c36892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.547593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77f589-30a3-46f5-817b-8d9db3c36892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:15.549245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77f589-30a3-46f5-817b-8d9db3c36892 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:15.561310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1108/2203] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:15.565563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f7493-3bfd-4674-8fc2-236f8b3c65ad admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:15.575988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f7493-3bfd-4674-8fc2-236f8b3c65ad admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:15.580547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e102c68-0322-4f36-813f-c27c60df352a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:15.581178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e102c68-0322-4f36-813f-c27c60df352a admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.581451 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e102c68-0322-4f36-813f-c27c60df352a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.581722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e102c68-0322-4f36-813f-c27c60df352a admin admin] PATH_INFO: `/v3/projects/f5454c437c7b458393bfc67a0b5108f7/users/bc8445d9d23443e2be4d1776e7858473/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.596305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e102c68-0322-4f36-813f-c27c60df352a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5454c437c7b458393bfc67a0b5108f7, user_id=bc8445d9d23443e2be4d1776e7858473, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:15.599018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e102c68-0322-4f36-813f-c27c60df352a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:15.611647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1096/2204] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:15 2022] PUT /identity/v3/projects/f5454c437c7b458393bfc67a0b5108f7/users/bc8445d9d23443e2be4d1776e7858473/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:15.615480 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77f589-30a3-46f5-817b-8d9db3c36892 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:15.626080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77f589-30a3-46f5-817b-8d9db3c36892 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:15.681456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa2f90-ad06-4165-9334-a9ddbba455c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:15.682048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa2f90-ad06-4165-9334-a9ddbba455c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.682326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa2f90-ad06-4165-9334-a9ddbba455c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.682590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa2f90-ad06-4165-9334-a9ddbba455c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.683241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa2f90-ad06-4165-9334-a9ddbba455c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:15.685424 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa2f90-ad06-4165-9334-a9ddbba455c7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:15.695108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1109/2205] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:15.699325 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e102c68-0322-4f36-813f-c27c60df352a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:15.709329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e102c68-0322-4f36-813f-c27c60df352a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:15.713839 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffbbfe-2585-4d10-b2c0-3e71987f9bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:15.714637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffbbfe-2585-4d10-b2c0-3e71987f9bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.714789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffbbfe-2585-4d10-b2c0-3e71987f9bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.715103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffbbfe-2585-4d10-b2c0-3e71987f9bd3 admin admin] PATH_INFO: `/v3/projects/f5454c437c7b458393bfc67a0b5108f7/users/bc8445d9d23443e2be4d1776e7858473/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.716972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffbbfe-2585-4d10-b2c0-3e71987f9bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5454c437c7b458393bfc67a0b5108f7, user_id=bc8445d9d23443e2be4d1776e7858473, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:15.719678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffbbfe-2585-4d10-b2c0-3e71987f9bd3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:15.729513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1097/2206] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:15 2022] PUT /identity/v3/projects/f5454c437c7b458393bfc67a0b5108f7/users/bc8445d9d23443e2be4d1776e7858473/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:15.740844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e3a60-8c30-4686-8ddc-fa22ebc9ab3d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.741128 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e3a60-8c30-4686-8ddc-fa22ebc9ab3d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.741401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e3a60-8c30-4686-8ddc-fa22ebc9ab3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.768920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-863e3a60-8c30-4686-8ddc-fa22ebc9ab3d None None] MFA Rules not processed for user `bc8445d9d23443e2be4d1776e7858473`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:15.770123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-863e3a60-8c30-4686-8ddc-fa22ebc9ab3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:15.775014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:15.837069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1110/2207] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:15.849845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffbbfe-2585-4d10-b2c0-3e71987f9bd3 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:15.851551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c6ffbbfe-2585-4d10-b2c0-3e71987f9bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:15.858492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:15.909584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffbbfe-2585-4d10-b2c0-3e71987f9bd3 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:15.913134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11a9b0-2e40-4fc6-aee8-26f5a553222f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:15.913720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11a9b0-2e40-4fc6-aee8-26f5a553222f service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:15.913994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11a9b0-2e40-4fc6-aee8-26f5a553222f service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:15.914264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11a9b0-2e40-4fc6-aee8-26f5a553222f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:15.922934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11a9b0-2e40-4fc6-aee8-26f5a553222f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:15.924359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11a9b0-2e40-4fc6-aee8-26f5a553222f service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:15.933480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1098/2208] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:16.072133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e3a60-8c30-4686-8ddc-fa22ebc9ab3d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:16.081658 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e3a60-8c30-4686-8ddc-fa22ebc9ab3d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:16.085372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe059-e448-48f0-9913-921ea26385b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:16.085972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe059-e448-48f0-9913-921ea26385b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:16.086249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe059-e448-48f0-9913-921ea26385b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:16.086521 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe059-e448-48f0-9913-921ea26385b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:16.144240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fe059-e448-48f0-9913-921ea26385b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:16.145917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fe059-e448-48f0-9913-921ea26385b2 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:16.155711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1111/2209] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:04:16.467416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11a9b0-2e40-4fc6-aee8-26f5a553222f service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:16.476650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11a9b0-2e40-4fc6-aee8-26f5a553222f service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:16.481148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:16.481753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:16.482028 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:16.482300 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:16.484148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:16.565400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:16.567106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:16.581523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1099/2210] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:04:18.684594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe059-e448-48f0-9913-921ea26385b2 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:18.693171 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe059-e448-48f0-9913-921ea26385b2 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:18.696948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b75a36-231e-427c-ac3d-f280d140c6fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:18.697556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b75a36-231e-427c-ac3d-f280d140c6fb service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:18.697832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b75a36-231e-427c-ac3d-f280d140c6fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:18.698100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b75a36-231e-427c-ac3d-f280d140c6fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:18.700502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-87b75a36-231e-427c-ac3d-f280d140c6fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:18.797133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b75a36-231e-427c-ac3d-f280d140c6fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:18.798696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b75a36-231e-427c-ac3d-f280d140c6fb service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:18.814023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1112/2211] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:24.114552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:24.116144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:24.196592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb7071-1add-4927-ba58-4bcaa1f2762a service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:24.202987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:24.203161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:24.203394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:24.203666 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] PATH_INFO: `/v3/users/4736989f3b544a91919a4852cabe7e20` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:24.204784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4736989f3b544a91919a4852cabe7e20)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:24.207185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:24.285431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4736989f3b544a91919a4852cabe7e20'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:24.288987 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:24.296116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4736989f3b544a91919a4852cabe7e20'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:24.305506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4736989f3b544a91919a4852cabe7e20'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:24.366758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4736989f3b544a91919a4852cabe7e20'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:24.370291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4736989f3b544a91919a4852cabe7e20'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:24.385901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1100/2212] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:24 2022] DELETE /identity/v3/users/4736989f3b544a91919a4852cabe7e20 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:24.391286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b71b61-cdef-4e25-a226-67db6618a8d5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:24.391801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b71b61-cdef-4e25-a226-67db6618a8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:24.391882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b71b61-cdef-4e25-a226-67db6618a8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:24.430760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e1b71b61-cdef-4e25-a226-67db6618a8d5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:24.432427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e1b71b61-cdef-4e25-a226-67db6618a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:24.470356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:24.507589 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1113/2213] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:24.530063 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:24.531937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:24.606807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862b62d-cfbf-4c60-9f1b-deeb8c7c58a7 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:24.610688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dec035-29a5-4fec-85b6-2b9f486f9589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:24.611620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dec035-29a5-4fec-85b6-2b9f486f9589 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:24.612101 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dec035-29a5-4fec-85b6-2b9f486f9589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:24.612474 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dec035-29a5-4fec-85b6-2b9f486f9589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:24.623328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dec035-29a5-4fec-85b6-2b9f486f9589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:24.624857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dec035-29a5-4fec-85b6-2b9f486f9589 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:24.635700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1101/2214] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:24.872501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b71b61-cdef-4e25-a226-67db6618a8d5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:24.873932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e1b71b61-cdef-4e25-a226-67db6618a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:24.888460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b71b61-cdef-4e25-a226-67db6618a8d5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:24.894276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:24.894925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:24.895396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:24.895676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] PATH_INFO: `/v3/users/ed687d192fbf4bde9b5c0349b1ef7ac8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:24.897708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed687d192fbf4bde9b5c0349b1ef7ac8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:24.899936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:24.960273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed687d192fbf4bde9b5c0349b1ef7ac8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:24.990163 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed687d192fbf4bde9b5c0349b1ef7ac8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.014876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed687d192fbf4bde9b5c0349b1ef7ac8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.015785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed687d192fbf4bde9b5c0349b1ef7ac8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.016773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed687d192fbf4bde9b5c0349b1ef7ac8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.057944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1114/2215] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:24 2022] DELETE /identity/v3/users/ed687d192fbf4bde9b5c0349b1ef7ac8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:25.058332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6013d-334a-4ebe-a724-0989e068a59e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:25.058623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6013d-334a-4ebe-a724-0989e068a59e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:25.059461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6013d-334a-4ebe-a724-0989e068a59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:25.092549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-33c6013d-334a-4ebe-a724-0989e068a59e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:25.094190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-33c6013d-334a-4ebe-a724-0989e068a59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:25.202516 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1102/2216] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:25.207751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:25.212969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:25.282747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6013d-334a-4ebe-a724-0989e068a59e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:25.285644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-33c6013d-334a-4ebe-a724-0989e068a59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:25.308420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6013d-334a-4ebe-a724-0989e068a59e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:25.327806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:25.328640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:25.328918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:25.329316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] PATH_INFO: `/v3/projects/187a2119d50c4d7f8cd7c420c2969595` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:25.330636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=187a2119d50c4d7f8cd7c420c2969595)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:25.331385 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a63c5-07ac-49e1-8d0d-c8f5f0cdfc78 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:25.332768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:25.335384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82872d-0957-408e-b757-205f92fde452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:25.336027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82872d-0957-408e-b757-205f92fde452 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:25.336306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82872d-0957-408e-b757-205f92fde452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:25.336600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82872d-0957-408e-b757-205f92fde452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:25.347244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82872d-0957-408e-b757-205f92fde452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:25.348846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82872d-0957-408e-b757-205f92fde452 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:25.360242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1115/2217] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:25.406869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '187a2119d50c4d7f8cd7c420c2969595', 'request_id': 'req-7cd50bb0-88b2-46b9-882d-777c4d7328e9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.423753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '187a2119d50c4d7f8cd7c420c2969595', 'request_id': 'req-7cd50bb0-88b2-46b9-882d-777c4d7328e9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.438505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '187a2119d50c4d7f8cd7c420c2969595', 'request_id': 'req-7cd50bb0-88b2-46b9-882d-777c4d7328e9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.444478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:25.465995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] The token cache is being invalidate because project 187a2119d50c4d7f8cd7c420c2969595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:25.466585 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.468993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7cd50bb0-88b2-46b9-882d-777c4d7328e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.477074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1103/2218] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:25 2022] DELETE /identity/v3/projects/187a2119d50c4d7f8cd7c420c2969595 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:25.478904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fffabe-9d3a-4730-a981-53f6f38b35cd None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:25.479457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fffabe-9d3a-4730-a981-53f6f38b35cd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:25.479739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fffabe-9d3a-4730-a981-53f6f38b35cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:25.506518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-93fffabe-9d3a-4730-a981-53f6f38b35cd None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:25.508110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-93fffabe-9d3a-4730-a981-53f6f38b35cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:25.584636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:25.608303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1116/2219] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:25.618808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c233c-a257-4215-a489-e5b44bf627ac None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:25.620599 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c233c-a257-4215-a489-e5b44bf627ac None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:25.620997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c233c-a257-4215-a489-e5b44bf627ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:25.687684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-609c233c-a257-4215-a489-e5b44bf627ac None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:25.688904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-609c233c-a257-4215-a489-e5b44bf627ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:25.702342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fffabe-9d3a-4730-a981-53f6f38b35cd None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:25.707356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1104/2220] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:25.712965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fffabe-9d3a-4730-a981-53f6f38b35cd None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:25.720872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:25.721504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:25.721828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:25.722107 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:25.722869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:04:25.723213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:25.725085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:25.766239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1117/2221] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:04:25.768387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c233c-a257-4215-a489-e5b44bf627ac None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:25.772663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:25.774509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:25.782236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c233c-a257-4215-a489-e5b44bf627ac None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:25.791223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ed51-0b30-4bc3-959e-2b5020ce04f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:25.792233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ed51-0b30-4bc3-959e-2b5020ce04f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:25.792761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ed51-0b30-4bc3-959e-2b5020ce04f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:25.793614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ed51-0b30-4bc3-959e-2b5020ce04f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:25.794890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2ed51-0b30-4bc3-959e-2b5020ce04f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:25.798212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2ed51-0b30-4bc3-959e-2b5020ce04f7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:25.802231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e8640-be8e-4fe0-80a3-a23a4046f3ca admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:25.811838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:25.813029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:25.814561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:25.814885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] PATH_INFO: `/v3/projects/8a1f9a2e54fb44c1a5ad56cb1dd6b8f2` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:25.817402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1f9a2e54fb44c1a5ad56cb1dd6b8f2)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:25.821422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:25.866946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1105/2222] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:25 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:04:25.871667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ed51-0b30-4bc3-959e-2b5020ce04f7 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:25.933062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1f9a2e54fb44c1a5ad56cb1dd6b8f2', 'request_id': 'req-d079adaa-bed5-451c-9d13-916b6f022845'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.944137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1f9a2e54fb44c1a5ad56cb1dd6b8f2', 'request_id': 'req-d079adaa-bed5-451c-9d13-916b6f022845'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.955754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1f9a2e54fb44c1a5ad56cb1dd6b8f2', 'request_id': 'req-d079adaa-bed5-451c-9d13-916b6f022845'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.966789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] The token cache is being invalidate because project 8a1f9a2e54fb44c1a5ad56cb1dd6b8f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:25.967109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.967779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:25.970184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ed51-0b30-4bc3-959e-2b5020ce04f7 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:25.972460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1118/2223] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:25 2022] DELETE /identity/v3/projects/8a1f9a2e54fb44c1a5ad56cb1dd6b8f2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:25.973469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:25.975101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:26.040190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17bb06-ee57-4f26-bf2a-1109fce57928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:26.040864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17bb06-ee57-4f26-bf2a-1109fce57928 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.041206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17bb06-ee57-4f26-bf2a-1109fce57928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.041547 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17bb06-ee57-4f26-bf2a-1109fce57928 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.042365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17bb06-ee57-4f26-bf2a-1109fce57928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:26.042486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079adaa-bed5-451c-9d13-916b6f022845 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:26.045097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17bb06-ee57-4f26-bf2a-1109fce57928 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:26.047689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:26.048310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.048585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.048858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] PATH_INFO: `/v3/users/5ff6b544ab1447e29b99d81ffcab60e0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.049975 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff6b544ab1447e29b99d81ffcab60e0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:26.051920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:26.066881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1106/2224] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:25 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:04:26.070830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17bb06-ee57-4f26-bf2a-1109fce57928 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:26.072464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ce17bb06-ee57-4f26-bf2a-1109fce57928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:26.084419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17bb06-ee57-4f26-bf2a-1109fce57928 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:26.089382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd52b66-c2a7-425e-9bc4-3ffa6bc286bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:26.090019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd52b66-c2a7-425e-9bc4-3ffa6bc286bf admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.090785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd52b66-c2a7-425e-9bc4-3ffa6bc286bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.091093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd52b66-c2a7-425e-9bc4-3ffa6bc286bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.091729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd52b66-c2a7-425e-9bc4-3ffa6bc286bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:26.093314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd52b66-c2a7-425e-9bc4-3ffa6bc286bf admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:26.103117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1107/2225] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:26.107360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff6b544ab1447e29b99d81ffcab60e0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.108215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd52b66-c2a7-425e-9bc4-3ffa6bc286bf admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:26.115164 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff6b544ab1447e29b99d81ffcab60e0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.117603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd52b66-c2a7-425e-9bc4-3ffa6bc286bf admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:26.120443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:26.122332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73224a8e-3d60-493e-9ded-3ba5dc20acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:26.122919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73224a8e-3d60-493e-9ded-3ba5dc20acfe admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.123239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73224a8e-3d60-493e-9ded-3ba5dc20acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.123503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73224a8e-3d60-493e-9ded-3ba5dc20acfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.124133 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73224a8e-3d60-493e-9ded-3ba5dc20acfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:26.126039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73224a8e-3d60-493e-9ded-3ba5dc20acfe admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:26.129245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff6b544ab1447e29b99d81ffcab60e0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.130156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff6b544ab1447e29b99d81ffcab60e0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.131199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-37153179-cc7d-4521-8ca9-70c79f30745c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff6b544ab1447e29b99d81ffcab60e0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.137648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1108/2226] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:04:26.148081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73224a8e-3d60-493e-9ded-3ba5dc20acfe admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:26.151391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1119/2227] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:25 2022] DELETE /identity/v3/users/5ff6b544ab1447e29b99d81ffcab60e0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:26.152539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-73224a8e-3d60-493e-9ded-3ba5dc20acfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:26.168180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73224a8e-3d60-493e-9ded-3ba5dc20acfe admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:26.218334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:26.265361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f0779-c91d-454f-92e8-d918dedfc540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:26.266068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f0779-c91d-454f-92e8-d918dedfc540 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.266373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f0779-c91d-454f-92e8-d918dedfc540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.266676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f0779-c91d-454f-92e8-d918dedfc540 admin admin] PATH_INFO: `/v3/projects/281a4bf45d474217a84a31cee8128a87/users/e9b57c80a45a454680de3e69db92c6cc/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.284591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f0779-c91d-454f-92e8-d918dedfc540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281a4bf45d474217a84a31cee8128a87, user_id=e9b57c80a45a454680de3e69db92c6cc, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:26.287518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f0779-c91d-454f-92e8-d918dedfc540 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:26.303650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1109/2228] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:26 2022] PUT /identity/v3/projects/281a4bf45d474217a84a31cee8128a87/users/e9b57c80a45a454680de3e69db92c6cc/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:26.310155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7ab6e-a142-4396-87be-6ba4b63c728f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.310440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7ab6e-a142-4396-87be-6ba4b63c728f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.310711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7ab6e-a142-4396-87be-6ba4b63c728f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.331838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-82a7ab6e-a142-4396-87be-6ba4b63c728f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:26.333055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-82a7ab6e-a142-4396-87be-6ba4b63c728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:26.394770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1120/2229] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:26.405035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f0779-c91d-454f-92e8-d918dedfc540 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:26.406945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d79f0779-c91d-454f-92e8-d918dedfc540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:26.463055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7ab6e-a142-4396-87be-6ba4b63c728f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:26.464562 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-82a7ab6e-a142-4396-87be-6ba4b63c728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:26.473315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f0779-c91d-454f-92e8-d918dedfc540 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:26.478465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bbc3d-2d85-4a50-8466-5f96581427de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:26.479226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bbc3d-2d85-4a50-8466-5f96581427de service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.479404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7ab6e-a142-4396-87be-6ba4b63c728f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:26.479885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bbc3d-2d85-4a50-8466-5f96581427de service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.479885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bbc3d-2d85-4a50-8466-5f96581427de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.485021 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:26.485857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.486258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.486541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] PATH_INFO: `/v3/projects/2db23f650dce4b7bb3562e100e37ade2` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.488828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db23f650dce4b7bb3562e100e37ade2)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:26.489940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bbc3d-2d85-4a50-8466-5f96581427de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:26.490784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:26.491438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bbc3d-2d85-4a50-8466-5f96581427de service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:26.502174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1110/2230] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:26.583391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5d849-fda2-4202-be59-689f472aa747 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.583802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5d849-fda2-4202-be59-689f472aa747 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.584179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5d849-fda2-4202-be59-689f472aa747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.588496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db23f650dce4b7bb3562e100e37ade2', 'request_id': 'req-6f5b6827-dd76-448f-8efa-57b5c6c78beb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.604409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db23f650dce4b7bb3562e100e37ade2', 'request_id': 'req-6f5b6827-dd76-448f-8efa-57b5c6c78beb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.615495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-62f5d849-fda2-4202-be59-689f472aa747 None None] MFA Rules not processed for user `7ecb55aabbe64be7907123772c359287`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:26.616301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db23f650dce4b7bb3562e100e37ade2', 'request_id': 'req-6f5b6827-dd76-448f-8efa-57b5c6c78beb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.617242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-62f5d849-fda2-4202-be59-689f472aa747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:26.635794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] The token cache is being invalidate because project 2db23f650dce4b7bb3562e100e37ade2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:26.637702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.638486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.640296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1121/2231] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:26 2022] DELETE /identity/v3/projects/2db23f650dce4b7bb3562e100e37ade2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:26.644022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:26.647627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:26.722266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1111/2232] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:26.763503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b6827-dd76-448f-8efa-57b5c6c78beb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:26.771019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:26.771811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:26.772669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:26.773489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] PATH_INFO: `/v3/users/05138ae913904cbda172304be4c7a9dd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:26.774881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05138ae913904cbda172304be4c7a9dd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:26.777341 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:26.784023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:26.851392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05138ae913904cbda172304be4c7a9dd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.859032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05138ae913904cbda172304be4c7a9dd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.878332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05138ae913904cbda172304be4c7a9dd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.879410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05138ae913904cbda172304be4c7a9dd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.879950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05138ae913904cbda172304be4c7a9dd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:26.898116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1122/2233] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:26 2022] DELETE /identity/v3/users/05138ae913904cbda172304be4c7a9dd => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:27.243563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5d849-fda2-4202-be59-689f472aa747 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:27.258780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5d849-fda2-4202-be59-689f472aa747 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:27.343114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:27.344066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:27.344420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:27.344764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] PATH_INFO: `/v3/projects/812bdaacfcb242d7a8f099d53695f2b5` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:27.346034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=812bdaacfcb242d7a8f099d53695f2b5)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:27.348041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:27.359224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:27.408108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '812bdaacfcb242d7a8f099d53695f2b5', 'request_id': 'req-7f7b5b87-20ea-4876-8c48-37110c415b85'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:27.414886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '812bdaacfcb242d7a8f099d53695f2b5', 'request_id': 'req-7f7b5b87-20ea-4876-8c48-37110c415b85'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:27.423307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '812bdaacfcb242d7a8f099d53695f2b5', 'request_id': 'req-7f7b5b87-20ea-4876-8c48-37110c415b85'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:27.435194 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] The token cache is being invalidate because project 812bdaacfcb242d7a8f099d53695f2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:27.435515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:27.436171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7f7b5b87-20ea-4876-8c48-37110c415b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:27.438268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1112/2234] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:27 2022] DELETE /identity/v3/projects/812bdaacfcb242d7a8f099d53695f2b5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:28.387815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:28.389479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:28.412813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262c2f0-7147-4430-8292-b11b45a3da54 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:28.502861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:28.503883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:28.504318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:28.504686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] PATH_INFO: `/v3/users/53480141c7cf4aecb8c0653fd3cf30b5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:28.506122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53480141c7cf4aecb8c0653fd3cf30b5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:28.508384 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:28.612558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53480141c7cf4aecb8c0653fd3cf30b5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:28.628769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53480141c7cf4aecb8c0653fd3cf30b5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:28.651004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53480141c7cf4aecb8c0653fd3cf30b5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:28.652214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53480141c7cf4aecb8c0653fd3cf30b5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:28.653622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53480141c7cf4aecb8c0653fd3cf30b5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:28.671261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1123/2235] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:28 2022] DELETE /identity/v3/users/53480141c7cf4aecb8c0653fd3cf30b5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:28.675749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81832b-82b5-418c-807e-c9c35e780382 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:28.676697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81832b-82b5-418c-807e-c9c35e780382 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:28.677217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81832b-82b5-418c-807e-c9c35e780382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:28.726363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0f81832b-82b5-418c-807e-c9c35e780382 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:28.728830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0f81832b-82b5-418c-807e-c9c35e780382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:28.834081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1113/2236] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:28.880029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:28.882665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:28.889890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:29.046770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22e34-d984-4ff7-a72d-a6f4041a8c87 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:29.051248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9760774-d1f4-4ae5-ba8a-ab5914019e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:29.051961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9760774-d1f4-4ae5-ba8a-ab5914019e02 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:29.052284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9760774-d1f4-4ae5-ba8a-ab5914019e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:29.052583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9760774-d1f4-4ae5-ba8a-ab5914019e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:29.067666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9760774-d1f4-4ae5-ba8a-ab5914019e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:29.069380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9760774-d1f4-4ae5-ba8a-ab5914019e02 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:29.087347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1124/2237] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:30.491375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81832b-82b5-418c-807e-c9c35e780382 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:30.493312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0f81832b-82b5-418c-807e-c9c35e780382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:30.520246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81832b-82b5-418c-807e-c9c35e780382 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:30.525692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:30.526333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:30.526609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:30.526881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] PATH_INFO: `/v3/projects/ee7eab3a7c9a4405bbe490f0c9ca4a6c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:30.540727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7eab3a7c9a4405bbe490f0c9ca4a6c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:30.545271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:30.651374 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7eab3a7c9a4405bbe490f0c9ca4a6c', 'request_id': 'req-8566eb05-a2bd-489a-9225-fe9d346912ee'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.659565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7eab3a7c9a4405bbe490f0c9ca4a6c', 'request_id': 'req-8566eb05-a2bd-489a-9225-fe9d346912ee'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.671863 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7eab3a7c9a4405bbe490f0c9ca4a6c', 'request_id': 'req-8566eb05-a2bd-489a-9225-fe9d346912ee'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.699900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] The token cache is being invalidate because project ee7eab3a7c9a4405bbe490f0c9ca4a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:30.700491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.701378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.704398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1114/2238] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:30 2022] DELETE /identity/v3/projects/ee7eab3a7c9a4405bbe490f0c9ca4a6c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:30.708421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9760774-d1f4-4ae5-ba8a-ab5914019e02 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:30.732936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9760774-d1f4-4ae5-ba8a-ab5914019e02 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:30.815256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:30.815945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:30.816241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:30.816523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] PATH_INFO: `/v3/users/bc8445d9d23443e2be4d1776e7858473` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:30.817763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8445d9d23443e2be4d1776e7858473)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:30.821108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:30.879439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8445d9d23443e2be4d1776e7858473'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.887508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8445d9d23443e2be4d1776e7858473'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.904187 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8445d9d23443e2be4d1776e7858473'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.905115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8445d9d23443e2be4d1776e7858473'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.906155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-068feeaa-ea02-418b-8da2-3910da97c643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8445d9d23443e2be4d1776e7858473'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:30.922848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1125/2239] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:30 2022] DELETE /identity/v3/users/bc8445d9d23443e2be4d1776e7858473 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:04:31.244329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:31.245779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:31.259248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566eb05-a2bd-489a-9225-fe9d346912ee admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:31.279279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:31.319425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:31.320055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.320326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.320596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] PATH_INFO: `/v3/projects/f5454c437c7b458393bfc67a0b5108f7` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.321757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5454c437c7b458393bfc67a0b5108f7)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:31.323676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:31.384367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5454c437c7b458393bfc67a0b5108f7', 'request_id': 'req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:31.392939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5454c437c7b458393bfc67a0b5108f7', 'request_id': 'req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:31.421210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5454c437c7b458393bfc67a0b5108f7', 'request_id': 'req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:31.437825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] The token cache is being invalidate because project f5454c437c7b458393bfc67a0b5108f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:31.438527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:31.439479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f7638ccd-a0d3-4c3f-9dc9-f8114861c3b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:31.445733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1115/2240] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:31 2022] DELETE /identity/v3/projects/f5454c437c7b458393bfc67a0b5108f7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:31.447277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9677a36-f582-4c08-a25d-54b4e6ddd84c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.447747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9677a36-f582-4c08-a25d-54b4e6ddd84c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.448100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9677a36-f582-4c08-a25d-54b4e6ddd84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.469159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e9677a36-f582-4c08-a25d-54b4e6ddd84c None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:31.470448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e9677a36-f582-4c08-a25d-54b4e6ddd84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:31.495326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:31.555347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1126/2241] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:31.563134 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb2e97-dbc8-489f-a093-d2330a0398c7 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.564028 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb2e97-dbc8-489f-a093-d2330a0398c7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.564598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb2e97-dbc8-489f-a093-d2330a0398c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.588708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-82bb2e97-dbc8-489f-a093-d2330a0398c7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:31.590013 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-82bb2e97-dbc8-489f-a093-d2330a0398c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:31.599496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1116/2242] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:31.603879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9677a36-f582-4c08-a25d-54b4e6ddd84c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:31.614397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9677a36-f582-4c08-a25d-54b4e6ddd84c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:31.619213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedf32c-1693-48db-a62c-81b9462b4a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:31.619814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedf32c-1693-48db-a62c-81b9462b4a6d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.620092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedf32c-1693-48db-a62c-81b9462b4a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.620364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedf32c-1693-48db-a62c-81b9462b4a6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.621691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedf32c-1693-48db-a62c-81b9462b4a6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:04:31.622071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedf32c-1693-48db-a62c-81b9462b4a6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:31.623818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedf32c-1693-48db-a62c-81b9462b4a6d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:31.635791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1127/2243] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:04:31.640318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb2e97-dbc8-489f-a093-d2330a0398c7 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:31.649338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb2e97-dbc8-489f-a093-d2330a0398c7 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:31.654466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20eea05-fa72-41d7-aedd-3d02b4151597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:31.655149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20eea05-fa72-41d7-aedd-3d02b4151597 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.655434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20eea05-fa72-41d7-aedd-3d02b4151597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.655732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20eea05-fa72-41d7-aedd-3d02b4151597 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.656391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20eea05-fa72-41d7-aedd-3d02b4151597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:31.658697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20eea05-fa72-41d7-aedd-3d02b4151597 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:31.691686 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1117/2244] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:31 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:31.693374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedf32c-1693-48db-a62c-81b9462b4a6d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:31.703658 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedf32c-1693-48db-a62c-81b9462b4a6d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:31.765187 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bfddb-7d70-4531-ab58-c13c475603c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:31.765856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bfddb-7d70-4531-ab58-c13c475603c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.766066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bfddb-7d70-4531-ab58-c13c475603c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.766336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bfddb-7d70-4531-ab58-c13c475603c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.766987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bfddb-7d70-4531-ab58-c13c475603c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:31.768983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bfddb-7d70-4531-ab58-c13c475603c3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:31.792501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1128/2245] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:31 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:31.796033 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20eea05-fa72-41d7-aedd-3d02b4151597 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:31.808000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20eea05-fa72-41d7-aedd-3d02b4151597 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:31.817131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84d2f9-be46-4b2b-9a32-b575c89f1b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:31.817758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84d2f9-be46-4b2b-9a32-b575c89f1b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.818010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84d2f9-be46-4b2b-9a32-b575c89f1b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.818281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84d2f9-be46-4b2b-9a32-b575c89f1b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.818922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84d2f9-be46-4b2b-9a32-b575c89f1b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:31.820550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84d2f9-be46-4b2b-9a32-b575c89f1b16 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:31.835285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1118/2246] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:31.837999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bfddb-7d70-4531-ab58-c13c475603c3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:31.850073 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bfddb-7d70-4531-ab58-c13c475603c3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:31.857061 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490ff98-9d32-48f0-8794-a59682cdcc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:31.857679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490ff98-9d32-48f0-8794-a59682cdcc93 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.858100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490ff98-9d32-48f0-8794-a59682cdcc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.858379 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490ff98-9d32-48f0-8794-a59682cdcc93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.859066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490ff98-9d32-48f0-8794-a59682cdcc93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:31.860661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490ff98-9d32-48f0-8794-a59682cdcc93 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:31.876406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1129/2247] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:31.881621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84d2f9-be46-4b2b-9a32-b575c89f1b16 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:31.896636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84d2f9-be46-4b2b-9a32-b575c89f1b16 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:31.905234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97049326-0cbb-4d7a-afde-e069d0f06506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:31.905980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97049326-0cbb-4d7a-afde-e069d0f06506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.906265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97049326-0cbb-4d7a-afde-e069d0f06506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.906541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97049326-0cbb-4d7a-afde-e069d0f06506 admin admin] PATH_INFO: `/v3/projects/8c3cb6c32f2145588f909d472cae35e0/users/d8b92f9fcb6144a2b1c7d69afaa310da/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.926868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97049326-0cbb-4d7a-afde-e069d0f06506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3cb6c32f2145588f909d472cae35e0, user_id=d8b92f9fcb6144a2b1c7d69afaa310da, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:31.929724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97049326-0cbb-4d7a-afde-e069d0f06506 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:31.951651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e32799-c74c-4495-965e-537cd594e8f1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:31.952302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e32799-c74c-4495-965e-537cd594e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:31.952913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e32799-c74c-4495-965e-537cd594e8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:31.957273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1119/2248] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:31 2022] PUT /identity/v3/projects/8c3cb6c32f2145588f909d472cae35e0/users/d8b92f9fcb6144a2b1c7d69afaa310da/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:32.007136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-65e32799-c74c-4495-965e-537cd594e8f1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:32.009123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-65e32799-c74c-4495-965e-537cd594e8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:32.135104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1130/2249] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:32.168466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97049326-0cbb-4d7a-afde-e069d0f06506 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:32.170323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-97049326-0cbb-4d7a-afde-e069d0f06506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:32.260364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:32.279651 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97049326-0cbb-4d7a-afde-e069d0f06506 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:32.293693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7dc9-ff8a-493b-b092-441b7e7a78aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:32.294322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7dc9-ff8a-493b-b092-441b7e7a78aa service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:32.294600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7dc9-ff8a-493b-b092-441b7e7a78aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:32.295049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7dc9-ff8a-493b-b092-441b7e7a78aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:32.312051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe7dc9-ff8a-493b-b092-441b7e7a78aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:32.313682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe7dc9-ff8a-493b-b092-441b7e7a78aa service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:32.328289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1120/2250] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:35.023817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4fe74-1f14-493f-a395-31f3e8121161 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:35.025109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4fe74-1f14-493f-a395-31f3e8121161 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:35.025551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4fe74-1f14-493f-a395-31f3e8121161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:35.069862 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d2e4fe74-1f14-493f-a395-31f3e8121161 None None] MFA Rules not processed for user `d8b92f9fcb6144a2b1c7d69afaa310da`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:35.073184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d2e4fe74-1f14-493f-a395-31f3e8121161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:35.144456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:35.231721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1131/2251] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:35.236548 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7dc9-ff8a-493b-b092-441b7e7a78aa service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:35.247932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7dc9-ff8a-493b-b092-441b7e7a78aa service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:35.254381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b67d2-fbb2-4b21-a2bf-ef6e201450a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:35.255027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b67d2-fbb2-4b21-a2bf-ef6e201450a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:35.255330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b67d2-fbb2-4b21-a2bf-ef6e201450a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:35.255597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b67d2-fbb2-4b21-a2bf-ef6e201450a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:35.256577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b67d2-fbb2-4b21-a2bf-ef6e201450a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:35.264789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b67d2-fbb2-4b21-a2bf-ef6e201450a3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:35.295634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1121/2252] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:35 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:35.299847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4fe74-1f14-493f-a395-31f3e8121161 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:35.312103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4fe74-1f14-493f-a395-31f3e8121161 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:35.386256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc49b96-e2d4-4a58-801f-811fc6f80bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:35.387222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc49b96-e2d4-4a58-801f-811fc6f80bad admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:35.387510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc49b96-e2d4-4a58-801f-811fc6f80bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:35.387801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc49b96-e2d4-4a58-801f-811fc6f80bad admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:35.388588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc49b96-e2d4-4a58-801f-811fc6f80bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:35.390579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc49b96-e2d4-4a58-801f-811fc6f80bad admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:35.425189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1132/2253] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:35 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:04:35.426291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b67d2-fbb2-4b21-a2bf-ef6e201450a3 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:35.461204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b67d2-fbb2-4b21-a2bf-ef6e201450a3 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:35.466442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5e729-d2cb-4874-857e-2563de4f0361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:35.467119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e729-d2cb-4874-857e-2563de4f0361 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:35.467389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e729-d2cb-4874-857e-2563de4f0361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:35.467776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e729-d2cb-4874-857e-2563de4f0361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:35.468446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5e729-d2cb-4874-857e-2563de4f0361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:35.470497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5e729-d2cb-4874-857e-2563de4f0361 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:35.517557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1122/2254] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:35.522241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc49b96-e2d4-4a58-801f-811fc6f80bad admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:35.524338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c248b-59ca-45e9-af50-c832d440d4a3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:35.524836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c248b-59ca-45e9-af50-c832d440d4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:35.525196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c248b-59ca-45e9-af50-c832d440d4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:35.538928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc49b96-e2d4-4a58-801f-811fc6f80bad admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:35.553870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e5a92-82cd-493b-8430-64370a9acf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:35.554978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e5a92-82cd-493b-8430-64370a9acf96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:35.555282 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e5a92-82cd-493b-8430-64370a9acf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:35.555566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e5a92-82cd-493b-8430-64370a9acf96 admin admin] PATH_INFO: `/v3/projects/655214bf0d4e4b6989d114072302c6e1/users/85c3f26d56e0433691ebedbefa50f9db/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:35.561641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-1f0c248b-59ca-45e9-af50-c832d440d4a3 None None] MFA Rules not processed for user `e9b57c80a45a454680de3e69db92c6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:35.565880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1f0c248b-59ca-45e9-af50-c832d440d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:35.576404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e5a92-82cd-493b-8430-64370a9acf96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=655214bf0d4e4b6989d114072302c6e1, user_id=85c3f26d56e0433691ebedbefa50f9db, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:35.579596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e5a92-82cd-493b-8430-64370a9acf96 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:35.602848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1133/2255] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:35 2022] PUT /identity/v3/projects/655214bf0d4e4b6989d114072302c6e1/users/85c3f26d56e0433691ebedbefa50f9db/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:35.606645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e5a92-82cd-493b-8430-64370a9acf96 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:35.667421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1123/2256] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:35.668777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c248b-59ca-45e9-af50-c832d440d4a3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:35.670908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1f0c248b-59ca-45e9-af50-c832d440d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:35.688237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c248b-59ca-45e9-af50-c832d440d4a3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:35.706388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e5a92-82cd-493b-8430-64370a9acf96 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:35.715516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670084-683f-4dd9-9085-11d6b658cc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:35.716511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670084-683f-4dd9-9085-11d6b658cc09 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:35.716943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670084-683f-4dd9-9085-11d6b658cc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:35.718574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670084-683f-4dd9-9085-11d6b658cc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:35.720307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4670084-683f-4dd9-9085-11d6b658cc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:35.722332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4670084-683f-4dd9-9085-11d6b658cc09 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:35.755190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1134/2257] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:35.759527 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670084-683f-4dd9-9085-11d6b658cc09 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:35.780772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670084-683f-4dd9-9085-11d6b658cc09 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:35.795749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0590b-1424-474e-bb3e-61813cbc02e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:35.796421 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0590b-1424-474e-bb3e-61813cbc02e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:35.796744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0590b-1424-474e-bb3e-61813cbc02e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:35.797023 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0590b-1424-474e-bb3e-61813cbc02e5 admin admin] PATH_INFO: `/v3/projects/655214bf0d4e4b6989d114072302c6e1/users/85c3f26d56e0433691ebedbefa50f9db/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:35.799889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0590b-1424-474e-bb3e-61813cbc02e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=655214bf0d4e4b6989d114072302c6e1, user_id=85c3f26d56e0433691ebedbefa50f9db, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:35.802528 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0590b-1424-474e-bb3e-61813cbc02e5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:35.839404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1135/2258] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:35 2022] PUT /identity/v3/projects/655214bf0d4e4b6989d114072302c6e1/users/85c3f26d56e0433691ebedbefa50f9db/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:35.843264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df1e2f-15ca-4f64-a1e2-7e96d720a943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:35.843884 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df1e2f-15ca-4f64-a1e2-7e96d720a943 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:35.844160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df1e2f-15ca-4f64-a1e2-7e96d720a943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:35.844426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df1e2f-15ca-4f64-a1e2-7e96d720a943 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:35.847776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df1e2f-15ca-4f64-a1e2-7e96d720a943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:35.848157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df1e2f-15ca-4f64-a1e2-7e96d720a943 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:35.925007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1124/2259] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:35 2022] POST /identity/v3/projects => generated 385 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:35.929427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0590b-1424-474e-bb3e-61813cbc02e5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:35.958856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:36.085383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0590b-1424-474e-bb3e-61813cbc02e5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:36.191735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948028e8-a74e-4143-91ba-157f8eb99921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:36.192445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948028e8-a74e-4143-91ba-157f8eb99921 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:36.192803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948028e8-a74e-4143-91ba-157f8eb99921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:36.193150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948028e8-a74e-4143-91ba-157f8eb99921 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:36.193867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948028e8-a74e-4143-91ba-157f8eb99921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:36.203871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948028e8-a74e-4143-91ba-157f8eb99921 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:36.263483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1136/2260] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:35 2022] POST /identity/v3/users => generated 299 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:36.266090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df1e2f-15ca-4f64-a1e2-7e96d720a943 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:36.285643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df1e2f-15ca-4f64-a1e2-7e96d720a943 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:36.295887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762c54b-a753-44ee-97a3-d98ad0405bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:36.297740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c54b-a753-44ee-97a3-d98ad0405bbb admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:36.298400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c54b-a753-44ee-97a3-d98ad0405bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:36.298976 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c54b-a753-44ee-97a3-d98ad0405bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:36.300254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762c54b-a753-44ee-97a3-d98ad0405bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:36.302639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762c54b-a753-44ee-97a3-d98ad0405bbb admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:36.352507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1125/2261] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:36.357091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948028e8-a74e-4143-91ba-157f8eb99921 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:36.370667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948028e8-a74e-4143-91ba-157f8eb99921 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:36.376830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1316055-de5c-4776-95f8-290dcd6e2913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:36.377736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1316055-de5c-4776-95f8-290dcd6e2913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:36.378090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1316055-de5c-4776-95f8-290dcd6e2913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:36.378440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1316055-de5c-4776-95f8-290dcd6e2913 admin admin] PATH_INFO: `/v3/projects/21d02ef92a3a4da7b5ed3e589dbe78fd/users/a49c7a9c2d4b40a68d2e478519a05b4f/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:36.399795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1316055-de5c-4776-95f8-290dcd6e2913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d02ef92a3a4da7b5ed3e589dbe78fd, user_id=a49c7a9c2d4b40a68d2e478519a05b4f, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:36.411788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1316055-de5c-4776-95f8-290dcd6e2913 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:36.429643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762c54b-a753-44ee-97a3-d98ad0405bbb admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:36.430244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1137/2262] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:36 2022] PUT /identity/v3/projects/21d02ef92a3a4da7b5ed3e589dbe78fd/users/a49c7a9c2d4b40a68d2e478519a05b4f/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:36.443603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762c54b-a753-44ee-97a3-d98ad0405bbb admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:36.473357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:36.564977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c76a8-cf59-4737-9b54-c8c165e73fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:36.565707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c76a8-cf59-4737-9b54-c8c165e73fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:36.566060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c76a8-cf59-4737-9b54-c8c165e73fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:36.566425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c76a8-cf59-4737-9b54-c8c165e73fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:36.567236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c76a8-cf59-4737-9b54-c8c165e73fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:36.568986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c76a8-cf59-4737-9b54-c8c165e73fc9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:36.585479 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1126/2263] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:36.586854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1316055-de5c-4776-95f8-290dcd6e2913 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:36.599353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1316055-de5c-4776-95f8-290dcd6e2913 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:36.605154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dede9b-9f02-4e07-b240-e877c1df023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:36.606356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dede9b-9f02-4e07-b240-e877c1df023f admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:36.606753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dede9b-9f02-4e07-b240-e877c1df023f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:36.607101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dede9b-9f02-4e07-b240-e877c1df023f admin admin] PATH_INFO: `/v3/projects/21d02ef92a3a4da7b5ed3e589dbe78fd/users/a49c7a9c2d4b40a68d2e478519a05b4f/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:36.609860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dede9b-9f02-4e07-b240-e877c1df023f admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d02ef92a3a4da7b5ed3e589dbe78fd, user_id=a49c7a9c2d4b40a68d2e478519a05b4f, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:36.612578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dede9b-9f02-4e07-b240-e877c1df023f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:36.624239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1138/2264] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:36 2022] PUT /identity/v3/projects/21d02ef92a3a4da7b5ed3e589dbe78fd/users/a49c7a9c2d4b40a68d2e478519a05b4f/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:37.503954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca72f6-6e2d-4aa0-a7b9-1def8064411e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:37.504517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca72f6-6e2d-4aa0-a7b9-1def8064411e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:37.504621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca72f6-6e2d-4aa0-a7b9-1def8064411e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:37.583572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-50ca72f6-6e2d-4aa0-a7b9-1def8064411e None None] MFA Rules not processed for user `85c3f26d56e0433691ebedbefa50f9db`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:37.585801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-50ca72f6-6e2d-4aa0-a7b9-1def8064411e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:37.784495 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1127/2265] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:37.793681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dede9b-9f02-4e07-b240-e877c1df023f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:37.796042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-83dede9b-9f02-4e07-b240-e877c1df023f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:37.940743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dede9b-9f02-4e07-b240-e877c1df023f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:37.947240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5930f7-671f-4105-af2f-8f3c685b27b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:37.948067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5930f7-671f-4105-af2f-8f3c685b27b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:37.948401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5930f7-671f-4105-af2f-8f3c685b27b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:37.948678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5930f7-671f-4105-af2f-8f3c685b27b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:37.986401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:38.112214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5930f7-671f-4105-af2f-8f3c685b27b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:38.113705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5930f7-671f-4105-af2f-8f3c685b27b7 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:38.130347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1139/2266] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:38.751848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164972a1-16c4-463d-bf22-a5bc629ebcce None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:38.752688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164972a1-16c4-463d-bf22-a5bc629ebcce None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:38.753336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164972a1-16c4-463d-bf22-a5bc629ebcce None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:38.754874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1128/2267] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 10:04:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:04:38.811488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f78d78-ad40-4bac-8b51-9645f06c1325 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:38.811960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f78d78-ad40-4bac-8b51-9645f06c1325 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:38.812292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f78d78-ad40-4bac-8b51-9645f06c1325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:38.850849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-38f78d78-ad40-4bac-8b51-9645f06c1325 None None] MFA Rules not processed for user `d3d7b3cb498a4da18e203456e09b59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:38.852920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-38f78d78-ad40-4bac-8b51-9645f06c1325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:38.898167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164972a1-16c4-463d-bf22-a5bc629ebcce None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:38.904459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-164972a1-16c4-463d-bf22-a5bc629ebcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:39.013804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1140/2268] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 10:04:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:39.028644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f78d78-ad40-4bac-8b51-9645f06c1325 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:39.031332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-38f78d78-ad40-4bac-8b51-9645f06c1325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:39.032999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164972a1-16c4-463d-bf22-a5bc629ebcce None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:39.038793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:39.039535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:39.039822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:39.040069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] PATH_INFO: `/v3/users/0d53857c736c4a75abe0d4979517bc58` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:39.041514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d53857c736c4a75abe0d4979517bc58)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:39.044955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:39.053551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f78d78-ad40-4bac-8b51-9645f06c1325 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:39.061055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ec996-5b1e-47e9-9102-975f82fb37e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:39.062431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ec996-5b1e-47e9-9102-975f82fb37e3 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:39.063003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ec996-5b1e-47e9-9102-975f82fb37e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:39.063483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ec996-5b1e-47e9-9102-975f82fb37e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:39.083767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ec996-5b1e-47e9-9102-975f82fb37e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:39.085890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ec996-5b1e-47e9-9102-975f82fb37e3 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:39.101166 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1141/2269] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:04:39.101964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:39.104155 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d53857c736c4a75abe0d4979517bc58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.112888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d53857c736c4a75abe0d4979517bc58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.126039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d53857c736c4a75abe0d4979517bc58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.140735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d53857c736c4a75abe0d4979517bc58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.142575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-31dfe458-dda4-4732-b642-59aebd39480c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d53857c736c4a75abe0d4979517bc58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.153831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1129/2270] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:04:38 2022] DELETE /identity/v3/users/0d53857c736c4a75abe0d4979517bc58 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:39.468008 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ec996-5b1e-47e9-9102-975f82fb37e3 service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:39.492531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ec996-5b1e-47e9-9102-975f82fb37e3 service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:39.591878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:39.592574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:39.592920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:39.593727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] PATH_INFO: `/v3/projects/229ece368fc1458ba1c6cd94cb28f87a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:39.594867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=229ece368fc1458ba1c6cd94cb28f87a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:39.597302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:39.692696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:39.695322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229ece368fc1458ba1c6cd94cb28f87a', 'request_id': 'req-bada46c2-0a75-49b7-97f7-dacfb88ac49a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.707647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229ece368fc1458ba1c6cd94cb28f87a', 'request_id': 'req-bada46c2-0a75-49b7-97f7-dacfb88ac49a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.736211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229ece368fc1458ba1c6cd94cb28f87a', 'request_id': 'req-bada46c2-0a75-49b7-97f7-dacfb88ac49a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.769852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] The token cache is being invalidate because project 229ece368fc1458ba1c6cd94cb28f87a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:04:39.770157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.771469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:04:39.779085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1142/2271] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:04:39 2022] DELETE /identity/v3/projects/229ece368fc1458ba1c6cd94cb28f87a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:39.780778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b5050-8c01-442d-906d-d8503a33e38f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:39.781240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b5050-8c01-442d-906d-d8503a33e38f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:39.781337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b5050-8c01-442d-906d-d8503a33e38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:39.832142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-057b5050-8c01-442d-906d-d8503a33e38f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:39.833215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-057b5050-8c01-442d-906d-d8503a33e38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:40.000698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:40.001576 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1130/2272] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:04:40.002298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:40.010682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043264c4-2a11-48bd-84e9-f17d54943ba6 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.011103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043264c4-2a11-48bd-84e9-f17d54943ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.011405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043264c4-2a11-48bd-84e9-f17d54943ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.065396 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-043264c4-2a11-48bd-84e9-f17d54943ba6 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:40.068448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-043264c4-2a11-48bd-84e9-f17d54943ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:40.080910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1131/2273] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:40.085764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043264c4-2a11-48bd-84e9-f17d54943ba6 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:40.097973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043264c4-2a11-48bd-84e9-f17d54943ba6 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:40.112119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718fb2e-861f-488b-ad10-e3785d302268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:40.112972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718fb2e-861f-488b-ad10-e3785d302268 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.113658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718fb2e-861f-488b-ad10-e3785d302268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.113936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718fb2e-861f-488b-ad10-e3785d302268 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.115846 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718fb2e-861f-488b-ad10-e3785d302268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:04:40.116158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718fb2e-861f-488b-ad10-e3785d302268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:40.118047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718fb2e-861f-488b-ad10-e3785d302268 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:40.134991 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1132/2274] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:04:40.141076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718fb2e-861f-488b-ad10-e3785d302268 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:40.156879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718fb2e-861f-488b-ad10-e3785d302268 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:40.158185 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada46c2-0a75-49b7-97f7-dacfb88ac49a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:40.163532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee4c90-1af4-4039-afe3-fd233ea9ab93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:40.164165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee4c90-1af4-4039-afe3-fd233ea9ab93 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.164438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee4c90-1af4-4039-afe3-fd233ea9ab93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.165367 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee4c90-1af4-4039-afe3-fd233ea9ab93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.165695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee4c90-1af4-4039-afe3-fd233ea9ab93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:40.167059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032b1da-70cc-480a-b95c-4fa4b9983cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:40.168477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee4c90-1af4-4039-afe3-fd233ea9ab93 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:40.168617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032b1da-70cc-480a-b95c-4fa4b9983cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.168969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032b1da-70cc-480a-b95c-4fa4b9983cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.169404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032b1da-70cc-480a-b95c-4fa4b9983cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.171337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2032b1da-70cc-480a-b95c-4fa4b9983cd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:40.222282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1133/2275] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:04:40 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:40.225312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee4c90-1af4-4039-afe3-fd233ea9ab93 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:40.285708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032b1da-70cc-480a-b95c-4fa4b9983cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:40.289354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032b1da-70cc-480a-b95c-4fa4b9983cd9 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:40.317842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1143/2276] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:40.327656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee4c90-1af4-4039-afe3-fd233ea9ab93 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:40.333253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f016aa-d10e-4097-8a6b-5606dfbcac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:40.334111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f016aa-d10e-4097-8a6b-5606dfbcac7b admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.334543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f016aa-d10e-4097-8a6b-5606dfbcac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.334996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f016aa-d10e-4097-8a6b-5606dfbcac7b admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.335770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f016aa-d10e-4097-8a6b-5606dfbcac7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:40.338690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f016aa-d10e-4097-8a6b-5606dfbcac7b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:40.371609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1134/2277] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:04:40 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:04:40.374868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032b1da-70cc-480a-b95c-4fa4b9983cd9 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:40.395252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032b1da-70cc-480a-b95c-4fa4b9983cd9 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:40.417462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3288d74-916e-4076-8214-537030b89a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:40.418846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3288d74-916e-4076-8214-537030b89a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.419313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3288d74-916e-4076-8214-537030b89a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.419737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3288d74-916e-4076-8214-537030b89a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.420588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3288d74-916e-4076-8214-537030b89a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:40.422790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3288d74-916e-4076-8214-537030b89a9b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:40.439653 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1144/2278] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:40.441045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f016aa-d10e-4097-8a6b-5606dfbcac7b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:40.452743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f016aa-d10e-4097-8a6b-5606dfbcac7b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:40.457895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79953f3e-d335-4f27-bb13-39269a55e880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:40.458508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79953f3e-d335-4f27-bb13-39269a55e880 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.458782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79953f3e-d335-4f27-bb13-39269a55e880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.459089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79953f3e-d335-4f27-bb13-39269a55e880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.460167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79953f3e-d335-4f27-bb13-39269a55e880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:40.461355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79953f3e-d335-4f27-bb13-39269a55e880 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:40.468861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:40.475136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1135/2279] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:04:40.479093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3288d74-916e-4076-8214-537030b89a9b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:40.503927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3288d74-916e-4076-8214-537030b89a9b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:40.509288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cb9a5-fc57-4ad3-af57-67df79fd4aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:40.509960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cb9a5-fc57-4ad3-af57-67df79fd4aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.510278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cb9a5-fc57-4ad3-af57-67df79fd4aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.510645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cb9a5-fc57-4ad3-af57-67df79fd4aa1 admin admin] PATH_INFO: `/v3/projects/20760466eac94b87b14c279bba223b2d/users/2ef546ba84fa46d581ed6c82ec302e7d/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.534832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8cb9a5-fc57-4ad3-af57-67df79fd4aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20760466eac94b87b14c279bba223b2d, user_id=2ef546ba84fa46d581ed6c82ec302e7d, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:40.538895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8cb9a5-fc57-4ad3-af57-67df79fd4aa1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:40.596991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1145/2280] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:04:40 2022] PUT /identity/v3/projects/20760466eac94b87b14c279bba223b2d/users/2ef546ba84fa46d581ed6c82ec302e7d/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:04:40.602094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7701d2f-762f-47bc-862e-45b0dae4f26b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.602565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7701d2f-762f-47bc-862e-45b0dae4f26b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.603081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7701d2f-762f-47bc-862e-45b0dae4f26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.652782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d7701d2f-762f-47bc-862e-45b0dae4f26b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:40.655068 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d7701d2f-762f-47bc-862e-45b0dae4f26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:40.791605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1136/2281] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:40.812779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cb9a5-fc57-4ad3-af57-67df79fd4aa1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:40.815657 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1c8cb9a5-fc57-4ad3-af57-67df79fd4aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:40.925013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cb9a5-fc57-4ad3-af57-67df79fd4aa1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:40.930569 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfb11c-a11b-4306-b1ff-a22cdbeea36c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:40.931340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfb11c-a11b-4306-b1ff-a22cdbeea36c service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:40.931744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfb11c-a11b-4306-b1ff-a22cdbeea36c service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:40.932135 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfb11c-a11b-4306-b1ff-a22cdbeea36c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:40.945786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:40.950651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfb11c-a11b-4306-b1ff-a22cdbeea36c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:40.952721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfb11c-a11b-4306-b1ff-a22cdbeea36c service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:40.970461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1146/2282] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:43.655527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ab50d-be52-4cb9-9a9e-25033172666a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:43.656634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ab50d-be52-4cb9-9a9e-25033172666a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:43.657092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ab50d-be52-4cb9-9a9e-25033172666a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:43.776339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-8c3ab50d-be52-4cb9-9a9e-25033172666a None None] MFA Rules not processed for user `a49c7a9c2d4b40a68d2e478519a05b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:43.778257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8c3ab50d-be52-4cb9-9a9e-25033172666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:43.901043 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1137/2283] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:43.923703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfb11c-a11b-4306-b1ff-a22cdbeea36c service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:43.925360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7bbfb11c-a11b-4306-b1ff-a22cdbeea36c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:44.082759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfb11c-a11b-4306-b1ff-a22cdbeea36c service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:44.088217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8f456f0d15b4d068cfcd6476563247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:44.089133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:44.089550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:44.091000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:44.093048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:44.251854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:44.253511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:44.279327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1147/2284] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:44.519839 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aaa94-703e-4722-a20f-2d71a8615708 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:44.520058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aaa94-703e-4722-a20f-2d71a8615708 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:44.520407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aaa94-703e-4722-a20f-2d71a8615708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:44.564172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-af5aaa94-703e-4722-a20f-2d71a8615708 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:44.565489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-af5aaa94-703e-4722-a20f-2d71a8615708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:44.685748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1138/2285] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:04:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:44.742234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:44.743751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:44.876356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62f64c-73c9-4a83-9c2b-be0818df043e service glance] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:44.882937 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0564270-f3e2-49a1-8ea0-3422a3597898 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:44.884337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0564270-f3e2-49a1-8ea0-3422a3597898 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:44.884788 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0564270-f3e2-49a1-8ea0-3422a3597898 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:44.885218 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0564270-f3e2-49a1-8ea0-3422a3597898 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:44.899063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0564270-f3e2-49a1-8ea0-3422a3597898 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:44.901125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0564270-f3e2-49a1-8ea0-3422a3597898 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:44.911734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1148/2286] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:04:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:04:50.181847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f8e1d-0ede-4469-b165-51f5a395525b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:50.182765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f8e1d-0ede-4469-b165-51f5a395525b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:50.183853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f8e1d-0ede-4469-b165-51f5a395525b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:50.251432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-4b3f8e1d-0ede-4469-b165-51f5a395525b None None] MFA Rules not processed for user `2ef546ba84fa46d581ed6c82ec302e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:50.253054 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4b3f8e1d-0ede-4469-b165-51f5a395525b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:50.322574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:50.469913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1139/2287] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:50.478844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0564270-f3e2-49a1-8ea0-3422a3597898 service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:50.480407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a0564270-f3e2-49a1-8ea0-3422a3597898 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:50.550254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:04:50.671864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0564270-f3e2-49a1-8ea0-3422a3597898 service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:50.686578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9df6f-d446-4664-8c15-a92de7e6ed7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:50.687310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9df6f-d446-4664-8c15-a92de7e6ed7d service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:50.687634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9df6f-d446-4664-8c15-a92de7e6ed7d service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:50.690393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9df6f-d446-4664-8c15-a92de7e6ed7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:50.709102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9df6f-d446-4664-8c15-a92de7e6ed7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:50.710707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9df6f-d446-4664-8c15-a92de7e6ed7d service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:50.728777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1149/2288] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:04:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:04:53.760242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50986b24-23b3-4804-9391-9b1915aec4b8 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:53.760242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50986b24-23b3-4804-9391-9b1915aec4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:53.760929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50986b24-23b3-4804-9391-9b1915aec4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:53.791100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-50986b24-23b3-4804-9391-9b1915aec4b8 None None] MFA Rules not processed for user `2ef546ba84fa46d581ed6c82ec302e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:53.793611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-50986b24-23b3-4804-9391-9b1915aec4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:53.807145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1140/2289] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:04:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:56.409558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b4a6-a9f6-4e07-ba22-8725a09fd262 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:56.410622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b4a6-a9f6-4e07-ba22-8725a09fd262 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:56.411267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b4a6-a9f6-4e07-ba22-8725a09fd262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:56.463893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bf89b4a6-a9f6-4e07-ba22-8725a09fd262 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:04:56.467298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bf89b4a6-a9f6-4e07-ba22-8725a09fd262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:04:56.488022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1150/2290] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:04:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:04:56.509836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50986b24-23b3-4804-9391-9b1915aec4b8 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:04:56.552316 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50986b24-23b3-4804-9391-9b1915aec4b8 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:04:56.566629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d6b1-e1ab-470c-ba4b-a73248872db5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:04:56.567602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d6b1-e1ab-470c-ba4b-a73248872db5 service swift] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:04:56.568060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d6b1-e1ab-470c-ba4b-a73248872db5 service swift] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:04:56.568506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d6b1-e1ab-470c-ba4b-a73248872db5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:04:56.598709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531d6b1-e1ab-470c-ba4b-a73248872db5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:04:56.602268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531d6b1-e1ab-470c-ba4b-a73248872db5 service swift] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:04:56.655270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1141/2291] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:04:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:05:01.276890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89b4a6-a9f6-4e07-ba22-8725a09fd262 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:01.281655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bf89b4a6-a9f6-4e07-ba22-8725a09fd262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:01.304820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89b4a6-a9f6-4e07-ba22-8725a09fd262 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:01.311216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:01.314764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:01.315325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:01.315821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] PATH_INFO: `/v3/users/d8b92f9fcb6144a2b1c7d69afaa310da` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:01.317366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b92f9fcb6144a2b1c7d69afaa310da)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:01.319775 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:01.447507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b92f9fcb6144a2b1c7d69afaa310da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:01.485954 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b92f9fcb6144a2b1c7d69afaa310da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:01.556442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b92f9fcb6144a2b1c7d69afaa310da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:01.557196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b92f9fcb6144a2b1c7d69afaa310da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:01.557960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b92f9fcb6144a2b1c7d69afaa310da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:01.603071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1151/2292] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:01 2022] DELETE /identity/v3/users/d8b92f9fcb6144a2b1c7d69afaa310da => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:01.907938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d6b1-e1ab-470c-ba4b-a73248872db5 service swift] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:01.946420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d6b1-e1ab-470c-ba4b-a73248872db5 service swift] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:02.091930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:02.126450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:02.126815 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:02.127125 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] PATH_INFO: `/v3/projects/8c3cb6c32f2145588f909d472cae35e0` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:02.136954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c3cb6c32f2145588f909d472cae35e0)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:02.147302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:02.264263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c3cb6c32f2145588f909d472cae35e0', 'request_id': 'req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.277103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c3cb6c32f2145588f909d472cae35e0', 'request_id': 'req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.294721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c3cb6c32f2145588f909d472cae35e0', 'request_id': 'req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.320597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] The token cache is being invalidate because project 8c3cb6c32f2145588f909d472cae35e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:02.321152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.321895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.328210 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1142/2293] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:01 2022] DELETE /identity/v3/projects/8c3cb6c32f2145588f909d472cae35e0 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:02.331337 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:02.333744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:02.361006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299d0b8-c450-44f0-a958-6083f68833d6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:02.495766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:02.496966 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:02.497267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:02.497709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] PATH_INFO: `/v3/users/85c3f26d56e0433691ebedbefa50f9db` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:02.499124 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c3f26d56e0433691ebedbefa50f9db)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:02.501386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:02.536628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:02.621499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c3f26d56e0433691ebedbefa50f9db'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.635034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c3f26d56e0433691ebedbefa50f9db'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.654707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c3f26d56e0433691ebedbefa50f9db'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.657307 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c3f26d56e0433691ebedbefa50f9db'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.657945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-313e2604-927c-4e48-8e88-5110651f3f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c3f26d56e0433691ebedbefa50f9db'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:02.676222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1152/2294] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:02 2022] DELETE /identity/v3/users/85c3f26d56e0433691ebedbefa50f9db => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:03.077406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:03.079408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:03.106321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd2cf1-e2b4-40cb-9fca-45e7a3e6d7cd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:03.193340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:03.274198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:03.274841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:03.275161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:03.275430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] PATH_INFO: `/v3/projects/655214bf0d4e4b6989d114072302c6e1` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:03.277447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] RBAC: Authorizing `identity:delete_project(project_id=655214bf0d4e4b6989d114072302c6e1)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:03.279400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:03.420360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '655214bf0d4e4b6989d114072302c6e1', 'request_id': 'req-e56d98f9-4874-4261-b42d-1f233f53082f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:03.440563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '655214bf0d4e4b6989d114072302c6e1', 'request_id': 'req-e56d98f9-4874-4261-b42d-1f233f53082f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:03.458836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '655214bf0d4e4b6989d114072302c6e1', 'request_id': 'req-e56d98f9-4874-4261-b42d-1f233f53082f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:03.502775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] The token cache is being invalidate because project 655214bf0d4e4b6989d114072302c6e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:03.502775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:03.502775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-e56d98f9-4874-4261-b42d-1f233f53082f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:03.504830 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1143/2295] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:03 2022] DELETE /identity/v3/projects/655214bf0d4e4b6989d114072302c6e1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:03.510131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd68ea0-5f0e-487b-b451-c2d5af76fd2d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:03.511841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd68ea0-5f0e-487b-b451-c2d5af76fd2d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:03.512279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd68ea0-5f0e-487b-b451-c2d5af76fd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:03.551253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1bd68ea0-5f0e-487b-b451-c2d5af76fd2d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:03.555998 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1bd68ea0-5f0e-487b-b451-c2d5af76fd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:03.717368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1153/2296] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:03.719935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67563-f8a7-4224-baf1-366606da7847 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:03.720240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67563-f8a7-4224-baf1-366606da7847 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:03.720512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc67563-f8a7-4224-baf1-366606da7847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:03.761493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-1bc67563-f8a7-4224-baf1-366606da7847 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:03.764196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1bc67563-f8a7-4224-baf1-366606da7847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:03.791022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd68ea0-5f0e-487b-b451-c2d5af76fd2d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:03.801477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd68ea0-5f0e-487b-b451-c2d5af76fd2d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:03.802359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1144/2297] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:03.814730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fc0c0-98be-4333-8f23-a8659789b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:03.815488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc0c0-98be-4333-8f23-a8659789b82e admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:03.815777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc0c0-98be-4333-8f23-a8659789b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:03.816048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc0c0-98be-4333-8f23-a8659789b82e admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:03.816746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fc0c0-98be-4333-8f23-a8659789b82e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:05:03.817009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fc0c0-98be-4333-8f23-a8659789b82e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:03.818930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fc0c0-98be-4333-8f23-a8659789b82e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:03.850099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1154/2298] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:05:03.854941 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc67563-f8a7-4224-baf1-366606da7847 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:03.866517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc67563-f8a7-4224-baf1-366606da7847 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:03.875397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f58d52-10e5-4eb0-9ba9-0ad4bc33f0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:03.876408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f58d52-10e5-4eb0-9ba9-0ad4bc33f0fd admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:03.876785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f58d52-10e5-4eb0-9ba9-0ad4bc33f0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:03.877156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f58d52-10e5-4eb0-9ba9-0ad4bc33f0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:03.877895 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f58d52-10e5-4eb0-9ba9-0ad4bc33f0fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:03.880728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f58d52-10e5-4eb0-9ba9-0ad4bc33f0fd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:03.921696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fc0c0-98be-4333-8f23-a8659789b82e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:03.922077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1145/2299] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:05:03 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:03.932069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fc0c0-98be-4333-8f23-a8659789b82e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:04.045034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8402aa6-6c81-45a8-8dc8-61246b7debbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:04.047416 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8402aa6-6c81-45a8-8dc8-61246b7debbe admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:04.047988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8402aa6-6c81-45a8-8dc8-61246b7debbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:04.048986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8402aa6-6c81-45a8-8dc8-61246b7debbe admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:04.050363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8402aa6-6c81-45a8-8dc8-61246b7debbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:04.053585 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8402aa6-6c81-45a8-8dc8-61246b7debbe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:04.114495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1155/2300] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:05:03 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:05:04.117800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f58d52-10e5-4eb0-9ba9-0ad4bc33f0fd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:04.136030 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f58d52-10e5-4eb0-9ba9-0ad4bc33f0fd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:04.151076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fb352-263c-4388-8c67-1ced32357eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:04.155406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fb352-263c-4388-8c67-1ced32357eba admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:04.155972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fb352-263c-4388-8c67-1ced32357eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:04.156350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fb352-263c-4388-8c67-1ced32357eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:04.157062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559fb352-263c-4388-8c67-1ced32357eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:04.161188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559fb352-263c-4388-8c67-1ced32357eba admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:04.186299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1146/2301] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:05:04.190378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8402aa6-6c81-45a8-8dc8-61246b7debbe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:04.202610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:04.218627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8402aa6-6c81-45a8-8dc8-61246b7debbe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:04.234865 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71bb67-48b1-40c4-a5e7-80a61c12da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:04.235846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71bb67-48b1-40c4-a5e7-80a61c12da8b admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:04.237208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71bb67-48b1-40c4-a5e7-80a61c12da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:04.237689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71bb67-48b1-40c4-a5e7-80a61c12da8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:04.244950 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71bb67-48b1-40c4-a5e7-80a61c12da8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:04.251648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71bb67-48b1-40c4-a5e7-80a61c12da8b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:04.297910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1156/2302] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:04.299061 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fb352-263c-4388-8c67-1ced32357eba admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:04.330807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fb352-263c-4388-8c67-1ced32357eba admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:04.342492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f08b40-8bf8-404e-97e0-74d1970eda2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:04.343169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f08b40-8bf8-404e-97e0-74d1970eda2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:04.343491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f08b40-8bf8-404e-97e0-74d1970eda2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:04.343707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f08b40-8bf8-404e-97e0-74d1970eda2a admin admin] PATH_INFO: `/v3/projects/7e73f19a58a5425ea50c49ecf17bb167/users/8e7d91d4a3c0481798fe6864e999ac5f/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:04.366249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f08b40-8bf8-404e-97e0-74d1970eda2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e73f19a58a5425ea50c49ecf17bb167, user_id=8e7d91d4a3c0481798fe6864e999ac5f, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:04.371537 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f08b40-8bf8-404e-97e0-74d1970eda2a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:04.403376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ff1dd-51ac-4ae2-a96e-4a1407b5f0ec None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:04.403665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ff1dd-51ac-4ae2-a96e-4a1407b5f0ec None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:04.403948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ff1dd-51ac-4ae2-a96e-4a1407b5f0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:04.413635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1147/2303] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:04 2022] PUT /identity/v3/projects/7e73f19a58a5425ea50c49ecf17bb167/users/8e7d91d4a3c0481798fe6864e999ac5f/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:04.429088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f27e34-630a-4747-8ee2-aee80374ec36 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:04.429758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f27e34-630a-4747-8ee2-aee80374ec36 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:04.430182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f27e34-630a-4747-8ee2-aee80374ec36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:04.454610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-759ff1dd-51ac-4ae2-a96e-4a1407b5f0ec None None] MFA Rules not processed for user `e9b57c80a45a454680de3e69db92c6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:04.455752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-759ff1dd-51ac-4ae2-a96e-4a1407b5f0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:04.469656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f3f27e34-630a-4747-8ee2-aee80374ec36 None None] MFA Rules not processed for user `8e7d91d4a3c0481798fe6864e999ac5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:04.471245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f3f27e34-630a-4747-8ee2-aee80374ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:04.540278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:04.602052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1148/2304] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:05:04.604898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f27e34-630a-4747-8ee2-aee80374ec36 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:04.608376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f3f27e34-630a-4747-8ee2-aee80374ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:04.617347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1157/2305] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:04.761058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f27e34-630a-4747-8ee2-aee80374ec36 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:04.764147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eac6af-79c4-40c4-be66-6329793f709d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:04.767156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac6af-79c4-40c4-be66-6329793f709d service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:04.767156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac6af-79c4-40c4-be66-6329793f709d service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:04.767456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac6af-79c4-40c4-be66-6329793f709d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:04.789420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac6af-79c4-40c4-be66-6329793f709d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:04.795366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac6af-79c4-40c4-be66-6329793f709d service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:04.836200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1149/2306] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:04.840252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ff1dd-51ac-4ae2-a96e-4a1407b5f0ec None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:04.923601 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ff1dd-51ac-4ae2-a96e-4a1407b5f0ec None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:04.928274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:04.928976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:04.929319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:04.929816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] PATH_INFO: `/v3/users/8e7d91d4a3c0481798fe6864e999ac5f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:04.932358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7d91d4a3c0481798fe6864e999ac5f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:04.934889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:05.017669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7d91d4a3c0481798fe6864e999ac5f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:05.043722 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7d91d4a3c0481798fe6864e999ac5f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:05.064287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7d91d4a3c0481798fe6864e999ac5f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:05.065628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7d91d4a3c0481798fe6864e999ac5f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:05.066807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7d91d4a3c0481798fe6864e999ac5f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:05.092191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1158/2307] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:04 2022] DELETE /identity/v3/users/8e7d91d4a3c0481798fe6864e999ac5f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:05:05.097164 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c90e0f-d841-46f9-a0c7-874e93012112 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:05.098612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c90e0f-d841-46f9-a0c7-874e93012112 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:05.098612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c90e0f-d841-46f9-a0c7-874e93012112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:05.142278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a7c90e0f-d841-46f9-a0c7-874e93012112 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:05.143732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a7c90e0f-d841-46f9-a0c7-874e93012112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:05.252511 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1150/2308] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:05:05.278810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:05.280851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:05.314396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:05.403545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cac3f-9624-4bd1-8110-a79fc3ecb328 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:05.410926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddf7d8-044e-4033-b7e9-f83ce5754b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:05.411728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddf7d8-044e-4033-b7e9-f83ce5754b3a service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:05.411933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddf7d8-044e-4033-b7e9-f83ce5754b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:05.412371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddf7d8-044e-4033-b7e9-f83ce5754b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:05.439305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddf7d8-044e-4033-b7e9-f83ce5754b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:05.440891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddf7d8-044e-4033-b7e9-f83ce5754b3a service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:05.464661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1159/2309] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:05.966181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f09e7-0879-46ae-a2ae-c50f116388e7 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:05.966593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f09e7-0879-46ae-a2ae-c50f116388e7 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:05.967001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f09e7-0879-46ae-a2ae-c50f116388e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:06.021285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e10f09e7-0879-46ae-a2ae-c50f116388e7 None None] MFA Rules not processed for user `2dd3653b931643318f1a1b0820620277`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:06.022966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e10f09e7-0879-46ae-a2ae-c50f116388e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:06.164360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1151/2310] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:06.236965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddf7d8-044e-4033-b7e9-f83ce5754b3a service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:06.239106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-89ddf7d8-044e-4033-b7e9-f83ce5754b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:06.267920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddf7d8-044e-4033-b7e9-f83ce5754b3a service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:06.302133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:06.303250 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:06.303686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:06.305274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] PATH_INFO: `/v3/projects/7e73f19a58a5425ea50c49ecf17bb167` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:06.309511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e73f19a58a5425ea50c49ecf17bb167)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:06.311846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:06.404063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e73f19a58a5425ea50c49ecf17bb167', 'request_id': 'req-0a487e73-e70f-4268-9875-d5352e70318b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:06.414516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e73f19a58a5425ea50c49ecf17bb167', 'request_id': 'req-0a487e73-e70f-4268-9875-d5352e70318b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:06.422359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e73f19a58a5425ea50c49ecf17bb167', 'request_id': 'req-0a487e73-e70f-4268-9875-d5352e70318b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:06.436265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] The token cache is being invalidate because project 7e73f19a58a5425ea50c49ecf17bb167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:06.436613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:06.437214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0a487e73-e70f-4268-9875-d5352e70318b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:06.439253 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1160/2311] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:06 2022] DELETE /identity/v3/projects/7e73f19a58a5425ea50c49ecf17bb167 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:06.444471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21ecd0-785d-4bd5-8aa7-61db2a427050 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:06.444940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21ecd0-785d-4bd5-8aa7-61db2a427050 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:06.445279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21ecd0-785d-4bd5-8aa7-61db2a427050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:06.472853 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-af21ecd0-785d-4bd5-8aa7-61db2a427050 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:06.474341 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-af21ecd0-785d-4bd5-8aa7-61db2a427050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:06.516186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:06.558693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1152/2312] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:06.566054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a07b7-c869-4075-8a22-77a083152404 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:06.566544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a07b7-c869-4075-8a22-77a083152404 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:06.567097 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a07b7-c869-4075-8a22-77a083152404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:06.613689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a05a07b7-c869-4075-8a22-77a083152404 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:06.614496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a05a07b7-c869-4075-8a22-77a083152404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:06.637392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1161/2313] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:06.642354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21ecd0-785d-4bd5-8aa7-61db2a427050 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:06.666152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21ecd0-785d-4bd5-8aa7-61db2a427050 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:06.672905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b385c07-0e31-4cc6-b178-525b9aca1c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:06.673537 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b385c07-0e31-4cc6-b178-525b9aca1c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:06.673800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b385c07-0e31-4cc6-b178-525b9aca1c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:06.674051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b385c07-0e31-4cc6-b178-525b9aca1c7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:06.674748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b385c07-0e31-4cc6-b178-525b9aca1c7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:05:06.675169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b385c07-0e31-4cc6-b178-525b9aca1c7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:06.676896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b385c07-0e31-4cc6-b178-525b9aca1c7c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:06.712954 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1153/2314] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:05:06.715734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a07b7-c869-4075-8a22-77a083152404 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:06.732431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a07b7-c869-4075-8a22-77a083152404 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:06.738247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fc076-f0a9-46b4-8727-7b597d6759b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:06.738873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc076-f0a9-46b4-8727-7b597d6759b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:06.739431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc076-f0a9-46b4-8727-7b597d6759b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:06.739634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc076-f0a9-46b4-8727-7b597d6759b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:06.740369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fc076-f0a9-46b4-8727-7b597d6759b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:06.742273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fc076-f0a9-46b4-8727-7b597d6759b1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:06.813586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1162/2315] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:05:06 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:06.855493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b385c07-0e31-4cc6-b178-525b9aca1c7c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:06.893907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b385c07-0e31-4cc6-b178-525b9aca1c7c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:06.986553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0b4c8-bfb0-4220-85f4-edf784dd4925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:06.987216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0b4c8-bfb0-4220-85f4-edf784dd4925 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:06.987497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0b4c8-bfb0-4220-85f4-edf784dd4925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:06.987768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0b4c8-bfb0-4220-85f4-edf784dd4925 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:06.988454 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0b4c8-bfb0-4220-85f4-edf784dd4925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:06.990476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0b4c8-bfb0-4220-85f4-edf784dd4925 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:07.043679 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1154/2316] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:05:06 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:07.046141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fc076-f0a9-46b4-8727-7b597d6759b1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:07.058352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fc076-f0a9-46b4-8727-7b597d6759b1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:07.063651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11294497-6efb-4800-a4bd-d4bbd621b3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:07.064265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11294497-6efb-4800-a4bd-d4bbd621b3da admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:07.064576 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11294497-6efb-4800-a4bd-d4bbd621b3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:07.064803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11294497-6efb-4800-a4bd-d4bbd621b3da admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:07.065565 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11294497-6efb-4800-a4bd-d4bbd621b3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:07.067721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11294497-6efb-4800-a4bd-d4bbd621b3da admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:07.091699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1163/2317] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:07.092099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0b4c8-bfb0-4220-85f4-edf784dd4925 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:07.103520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0b4c8-bfb0-4220-85f4-edf784dd4925 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:07.109144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171f34a-a12b-432e-81bb-afde1cf84fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:07.109896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171f34a-a12b-432e-81bb-afde1cf84fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:07.110577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171f34a-a12b-432e-81bb-afde1cf84fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:07.111025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171f34a-a12b-432e-81bb-afde1cf84fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:07.111690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171f34a-a12b-432e-81bb-afde1cf84fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:07.114352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171f34a-a12b-432e-81bb-afde1cf84fc9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:07.141468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1155/2318] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:05:07.142221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11294497-6efb-4800-a4bd-d4bbd621b3da admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:07.155897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11294497-6efb-4800-a4bd-d4bbd621b3da admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:07.165606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc594e-1346-46a5-b5fb-e127b6784754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:07.166292 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc594e-1346-46a5-b5fb-e127b6784754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:07.166552 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc594e-1346-46a5-b5fb-e127b6784754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:07.166853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc594e-1346-46a5-b5fb-e127b6784754 admin admin] PATH_INFO: `/v3/projects/2007adb709d24d7b8d7009d67d9dc9f0/users/939bd61e6fcd4b0f9c9ca86c37287293/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:07.193906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc594e-1346-46a5-b5fb-e127b6784754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2007adb709d24d7b8d7009d67d9dc9f0, user_id=939bd61e6fcd4b0f9c9ca86c37287293, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:07.198922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc594e-1346-46a5-b5fb-e127b6784754 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:07.230814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1164/2319] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:07 2022] PUT /identity/v3/projects/2007adb709d24d7b8d7009d67d9dc9f0/users/939bd61e6fcd4b0f9c9ca86c37287293/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:05:07.231946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99af4f-cf9f-4d85-8b02-753156222cc1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:07.232784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99af4f-cf9f-4d85-8b02-753156222cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:07.232784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99af4f-cf9f-4d85-8b02-753156222cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:07.278007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-4a99af4f-cf9f-4d85-8b02-753156222cc1 None None] MFA Rules not processed for user `939bd61e6fcd4b0f9c9ca86c37287293`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:07.279771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4a99af4f-cf9f-4d85-8b02-753156222cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:07.501201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1156/2320] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:05:07.518469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc594e-1346-46a5-b5fb-e127b6784754 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:07.520499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-16dc594e-1346-46a5-b5fb-e127b6784754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:07.593568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:07.631530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc594e-1346-46a5-b5fb-e127b6784754 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:07.644275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54118200-1403-4f49-be9b-fc7487a3901b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:07.645217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54118200-1403-4f49-be9b-fc7487a3901b service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:07.645560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54118200-1403-4f49-be9b-fc7487a3901b service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:07.645837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54118200-1403-4f49-be9b-fc7487a3901b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:07.660433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54118200-1403-4f49-be9b-fc7487a3901b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:07.662245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54118200-1403-4f49-be9b-fc7487a3901b service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:07.687013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1165/2321] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:08.921882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb914-2bfa-4f70-9204-8361d498a211 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:08.923587 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb914-2bfa-4f70-9204-8361d498a211 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:08.923975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb914-2bfa-4f70-9204-8361d498a211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:08.967447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-66beb914-2bfa-4f70-9204-8361d498a211 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:08.968754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-66beb914-2bfa-4f70-9204-8361d498a211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:09.019649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:09.076305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1157/2322] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:05:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:09.084773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54118200-1403-4f49-be9b-fc7487a3901b service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:09.086449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-54118200-1403-4f49-be9b-fc7487a3901b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:09.180780 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54118200-1403-4f49-be9b-fc7487a3901b service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:09.188557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d62e84-eee2-4646-8e58-e12a0663672f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:09.190610 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d62e84-eee2-4646-8e58-e12a0663672f service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:09.191105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d62e84-eee2-4646-8e58-e12a0663672f service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:09.191844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d62e84-eee2-4646-8e58-e12a0663672f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:09.211215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d62e84-eee2-4646-8e58-e12a0663672f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:09.215035 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d62e84-eee2-4646-8e58-e12a0663672f service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:09.230813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1166/2323] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:05:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:05:19.060865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=11 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:19.225581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=40 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:24.212263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefb0ee-4551-494a-a95d-48358f0cc3a8 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:24.212843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefb0ee-4551-494a-a95d-48358f0cc3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:24.212843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefb0ee-4551-494a-a95d-48358f0cc3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:24.233503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-abefb0ee-4551-494a-a95d-48358f0cc3a8 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:24.234706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-abefb0ee-4551-494a-a95d-48358f0cc3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:24.242462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1158/2324] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 10 10:05:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:24.256291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d62e84-eee2-4646-8e58-e12a0663672f service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:24.273561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d62e84-eee2-4646-8e58-e12a0663672f service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:24.281672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bede4-fc71-4442-ba29-a28b5601aa20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:24.282708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bede4-fc71-4442-ba29-a28b5601aa20 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:24.283259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bede4-fc71-4442-ba29-a28b5601aa20 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:24.283817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bede4-fc71-4442-ba29-a28b5601aa20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:24.298590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bede4-fc71-4442-ba29-a28b5601aa20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:24.300959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bede4-fc71-4442-ba29-a28b5601aa20 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:24.314677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1167/2325] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:05:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:05:28.885988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefb0ee-4551-494a-a95d-48358f0cc3a8 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:28.887589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-abefb0ee-4551-494a-a95d-48358f0cc3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:28.955407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefb0ee-4551-494a-a95d-48358f0cc3a8 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:28.974743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:28.975342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:28.975619 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:28.975892 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] PATH_INFO: `/v3/users/7ecb55aabbe64be7907123772c359287` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:28.977565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ecb55aabbe64be7907123772c359287)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:28.980706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:29.032460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ecb55aabbe64be7907123772c359287'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.042031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ecb55aabbe64be7907123772c359287'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.051174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ecb55aabbe64be7907123772c359287'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.065573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ecb55aabbe64be7907123772c359287'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.066306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bd169c5b-253c-4e37-b4e4-0cc07cdadb1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ecb55aabbe64be7907123772c359287'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.072405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1159/2326] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:28 2022] DELETE /identity/v3/users/7ecb55aabbe64be7907123772c359287 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:29.274409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bede4-fc71-4442-ba29-a28b5601aa20 service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:29.285007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bede4-fc71-4442-ba29-a28b5601aa20 service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:29.344069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:29.344697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.344978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.345241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] PATH_INFO: `/v3/projects/76f7f3766f4a44c4b3859b8199a670da` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:29.346361 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f7f3766f4a44c4b3859b8199a670da)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:29.348276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:29.417099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f7f3766f4a44c4b3859b8199a670da', 'request_id': 'req-9522a995-e5b7-49db-ad19-f31c1e486821'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.430485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f7f3766f4a44c4b3859b8199a670da', 'request_id': 'req-9522a995-e5b7-49db-ad19-f31c1e486821'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.442702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f7f3766f4a44c4b3859b8199a670da', 'request_id': 'req-9522a995-e5b7-49db-ad19-f31c1e486821'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.457345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] The token cache is being invalidate because project 76f7f3766f4a44c4b3859b8199a670da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:29.457345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.458396 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9522a995-e5b7-49db-ad19-f31c1e486821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:29.459948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1168/2327] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:29 2022] DELETE /identity/v3/projects/76f7f3766f4a44c4b3859b8199a670da => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:29.465287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a45d21-ad87-42ba-a18d-0c4b429b828b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.465550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a45d21-ad87-42ba-a18d-0c4b429b828b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.465817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a45d21-ad87-42ba-a18d-0c4b429b828b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:29.485563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e9a45d21-ad87-42ba-a18d-0c4b429b828b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:29.486734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e9a45d21-ad87-42ba-a18d-0c4b429b828b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:29.546338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1160/2328] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:29.554182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec777e5-5ce9-4a82-8c57-6cc0095fb12a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.554459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec777e5-5ce9-4a82-8c57-6cc0095fb12a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.554729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec777e5-5ce9-4a82-8c57-6cc0095fb12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:29.574871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cec777e5-5ce9-4a82-8c57-6cc0095fb12a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:29.576089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cec777e5-5ce9-4a82-8c57-6cc0095fb12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:29.584580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1169/2329] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:29.590790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a45d21-ad87-42ba-a18d-0c4b429b828b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:29.602109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a45d21-ad87-42ba-a18d-0c4b429b828b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:29.608016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3657-3eb1-4a6c-8c3b-9240bcc9a033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:29.608662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3657-3eb1-4a6c-8c3b-9240bcc9a033 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.608953 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3657-3eb1-4a6c-8c3b-9240bcc9a033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.609208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3657-3eb1-4a6c-8c3b-9240bcc9a033 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:29.609877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f3657-3eb1-4a6c-8c3b-9240bcc9a033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:05:29.610239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f3657-3eb1-4a6c-8c3b-9240bcc9a033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:29.611917 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f3657-3eb1-4a6c-8c3b-9240bcc9a033 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:29.620919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:29.625025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1161/2330] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:05:29.629877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec777e5-5ce9-4a82-8c57-6cc0095fb12a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:29.640836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec777e5-5ce9-4a82-8c57-6cc0095fb12a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:29.645839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7d4cb-d007-4670-aed5-0675c4f14f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:29.646575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7d4cb-d007-4670-aed5-0675c4f14f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.646925 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7d4cb-d007-4670-aed5-0675c4f14f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.647299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7d4cb-d007-4670-aed5-0675c4f14f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:29.648101 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7d4cb-d007-4670-aed5-0675c4f14f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:29.650283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7d4cb-d007-4670-aed5-0675c4f14f4c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:29.684156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1170/2331] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:05:29 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:29.688721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3657-3eb1-4a6c-8c3b-9240bcc9a033 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:29.700228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3657-3eb1-4a6c-8c3b-9240bcc9a033 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:29.784766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2721c1b-427b-4e36-85eb-d9d9a2cfabc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:29.788019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2721c1b-427b-4e36-85eb-d9d9a2cfabc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.791463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2721c1b-427b-4e36-85eb-d9d9a2cfabc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.793340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2721c1b-427b-4e36-85eb-d9d9a2cfabc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:29.794087 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2721c1b-427b-4e36-85eb-d9d9a2cfabc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:29.796381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2721c1b-427b-4e36-85eb-d9d9a2cfabc2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:29.831446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1162/2332] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:05:29 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:29.835594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7d4cb-d007-4670-aed5-0675c4f14f4c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:29.847898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7d4cb-d007-4670-aed5-0675c4f14f4c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:29.852867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9d855-0a8b-41d2-8c39-f53597263723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:29.853565 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9d855-0a8b-41d2-8c39-f53597263723 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.854588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9d855-0a8b-41d2-8c39-f53597263723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.854992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9d855-0a8b-41d2-8c39-f53597263723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:29.855856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9d855-0a8b-41d2-8c39-f53597263723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:29.857558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9d855-0a8b-41d2-8c39-f53597263723 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:29.868181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1171/2333] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:29.872974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2721c1b-427b-4e36-85eb-d9d9a2cfabc2 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:29.888556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2721c1b-427b-4e36-85eb-d9d9a2cfabc2 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:29.893738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd4bac-d18a-461d-a89d-a2bd1a01a4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:29.894427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd4bac-d18a-461d-a89d-a2bd1a01a4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.894774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd4bac-d18a-461d-a89d-a2bd1a01a4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.895179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd4bac-d18a-461d-a89d-a2bd1a01a4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:29.895893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd4bac-d18a-461d-a89d-a2bd1a01a4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:29.897913 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd4bac-d18a-461d-a89d-a2bd1a01a4f2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:29.918334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1163/2334] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:29.920259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9d855-0a8b-41d2-8c39-f53597263723 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:29.935613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9d855-0a8b-41d2-8c39-f53597263723 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:29.940599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c133a-c56b-4bac-93aa-8162b1a50b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:29.941376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c133a-c56b-4bac-93aa-8162b1a50b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.941662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c133a-c56b-4bac-93aa-8162b1a50b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.941962 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c133a-c56b-4bac-93aa-8162b1a50b34 admin admin] PATH_INFO: `/v3/projects/cac5c9eec1e14961a8a4863f7f6cadaf/users/9037cc040d2043c599e6e24c7247946c/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:29.958655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c133a-c56b-4bac-93aa-8162b1a50b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac5c9eec1e14961a8a4863f7f6cadaf, user_id=9037cc040d2043c599e6e24c7247946c, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:29.961385 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c133a-c56b-4bac-93aa-8162b1a50b34 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:29.974319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1172/2335] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:29 2022] PUT /identity/v3/projects/cac5c9eec1e14961a8a4863f7f6cadaf/users/9037cc040d2043c599e6e24c7247946c/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:29.985646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffe101-b1d0-41ee-9dec-5e337de073ad None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:29.985924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffe101-b1d0-41ee-9dec-5e337de073ad None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:29.986193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffe101-b1d0-41ee-9dec-5e337de073ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:30.014044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-20ffe101-b1d0-41ee-9dec-5e337de073ad None None] MFA Rules not processed for user `9037cc040d2043c599e6e24c7247946c`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:30.015457 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-20ffe101-b1d0-41ee-9dec-5e337de073ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:30.089665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1164/2336] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:30.101740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c133a-c56b-4bac-93aa-8162b1a50b34 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:30.103545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d84c133a-c56b-4bac-93aa-8162b1a50b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:30.120798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:30.180752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c133a-c56b-4bac-93aa-8162b1a50b34 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:30.191652 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b447cf-4f16-4b71-950e-bd6679552fc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:30.192364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b447cf-4f16-4b71-950e-bd6679552fc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:30.192703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b447cf-4f16-4b71-950e-bd6679552fc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:30.193062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b447cf-4f16-4b71-950e-bd6679552fc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:30.203156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b447cf-4f16-4b71-950e-bd6679552fc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:30.204718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b447cf-4f16-4b71-950e-bd6679552fc8 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:30.216093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1173/2337] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:34.916765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffe101-b1d0-41ee-9dec-5e337de073ad None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:34.977334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffe101-b1d0-41ee-9dec-5e337de073ad None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:34.982788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:34.983583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:34.983975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:34.984256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] PATH_INFO: `/v3/users/9037cc040d2043c599e6e24c7247946c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:34.985403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9037cc040d2043c599e6e24c7247946c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:34.987359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:34.999338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:35.043742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9037cc040d2043c599e6e24c7247946c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:35.052383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9037cc040d2043c599e6e24c7247946c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:35.063142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9037cc040d2043c599e6e24c7247946c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:35.077664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9037cc040d2043c599e6e24c7247946c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:35.078376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9037cc040d2043c599e6e24c7247946c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:35.084562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1165/2338] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:34 2022] DELETE /identity/v3/users/9037cc040d2043c599e6e24c7247946c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:35.088945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3d786-b4fa-479b-a5ad-ae66bd3aafc1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:35.089440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3d786-b4fa-479b-a5ad-ae66bd3aafc1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:35.089604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3d786-b4fa-479b-a5ad-ae66bd3aafc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:35.108351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1eb3d786-b4fa-479b-a5ad-ae66bd3aafc1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:35.109545 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1eb3d786-b4fa-479b-a5ad-ae66bd3aafc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:35.169503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1174/2339] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:35.180403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:35.182566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:35.241591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0815d2-6d26-4380-a578-677fa3b8a99c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:35.245274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f0bf5-09d7-4860-bf59-eed3a133943a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:35.245873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f0bf5-09d7-4860-bf59-eed3a133943a service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:35.246153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f0bf5-09d7-4860-bf59-eed3a133943a service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:35.246425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f0bf5-09d7-4860-bf59-eed3a133943a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:35.255780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f0bf5-09d7-4860-bf59-eed3a133943a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:35.257237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f0bf5-09d7-4860-bf59-eed3a133943a service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:35.267011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1166/2340] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:36.027766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3d786-b4fa-479b-a5ad-ae66bd3aafc1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:36.029291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1eb3d786-b4fa-479b-a5ad-ae66bd3aafc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:36.047112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3d786-b4fa-479b-a5ad-ae66bd3aafc1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:36.053921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:36.054703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:36.055089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:36.055434 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] PATH_INFO: `/v3/projects/cac5c9eec1e14961a8a4863f7f6cadaf` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:36.056818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=cac5c9eec1e14961a8a4863f7f6cadaf)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:36.059148 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:36.171518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac5c9eec1e14961a8a4863f7f6cadaf', 'request_id': 'req-46379def-0717-41bb-aa4b-0578828cd2da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:36.181052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac5c9eec1e14961a8a4863f7f6cadaf', 'request_id': 'req-46379def-0717-41bb-aa4b-0578828cd2da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:36.188367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac5c9eec1e14961a8a4863f7f6cadaf', 'request_id': 'req-46379def-0717-41bb-aa4b-0578828cd2da'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:36.199616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] The token cache is being invalidate because project cac5c9eec1e14961a8a4863f7f6cadaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:36.199903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:36.200588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-46379def-0717-41bb-aa4b-0578828cd2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:36.202326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1175/2341] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:36 2022] DELETE /identity/v3/projects/cac5c9eec1e14961a8a4863f7f6cadaf => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:38.049952 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f0bf5-09d7-4860-bf59-eed3a133943a service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:38.051401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4e8f0bf5-09d7-4860-bf59-eed3a133943a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:38.064270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f0bf5-09d7-4860-bf59-eed3a133943a service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:38.118919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:38.120155 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:38.120431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:38.120702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] PATH_INFO: `/v3/users/939bd61e6fcd4b0f9c9ca86c37287293` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:38.121813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=939bd61e6fcd4b0f9c9ca86c37287293)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:38.123762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:38.158045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:38.175505 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939bd61e6fcd4b0f9c9ca86c37287293'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.184805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939bd61e6fcd4b0f9c9ca86c37287293'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.192533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939bd61e6fcd4b0f9c9ca86c37287293'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.206064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939bd61e6fcd4b0f9c9ca86c37287293'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.206787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939bd61e6fcd4b0f9c9ca86c37287293'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.213462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1167/2342] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:38 2022] DELETE /identity/v3/users/939bd61e6fcd4b0f9c9ca86c37287293 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:38.217678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9d2a-2367-4e14-a622-6bef8d84aca8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:38.218149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9d2a-2367-4e14-a622-6bef8d84aca8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:38.218241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9d2a-2367-4e14-a622-6bef8d84aca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:38.237776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b86a9d2a-2367-4e14-a622-6bef8d84aca8 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:38.239251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b86a9d2a-2367-4e14-a622-6bef8d84aca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:38.275974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:38.305303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1176/2343] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:38.315155 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:38.316595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:38.403139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3f340-944c-4fea-b26c-f5533d9bc179 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:38.409827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45836fa0-bf7a-469b-a676-8b141edfe408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:38.410693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45836fa0-bf7a-469b-a676-8b141edfe408 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:38.411526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45836fa0-bf7a-469b-a676-8b141edfe408 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:38.411824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45836fa0-bf7a-469b-a676-8b141edfe408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:38.423470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45836fa0-bf7a-469b-a676-8b141edfe408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:38.425107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45836fa0-bf7a-469b-a676-8b141edfe408 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:38.435770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1168/2344] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:38.804331 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a9d2a-2367-4e14-a622-6bef8d84aca8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:38.805897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b86a9d2a-2367-4e14-a622-6bef8d84aca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:38.818359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a9d2a-2367-4e14-a622-6bef8d84aca8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:38.823858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:38.824575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:38.824853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:38.825129 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] PATH_INFO: `/v3/projects/2007adb709d24d7b8d7009d67d9dc9f0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:38.826245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2007adb709d24d7b8d7009d67d9dc9f0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:38.828272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:38.874349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2007adb709d24d7b8d7009d67d9dc9f0', 'request_id': 'req-2747c48b-28e9-442c-9249-ef63f6b83cc3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.882210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2007adb709d24d7b8d7009d67d9dc9f0', 'request_id': 'req-2747c48b-28e9-442c-9249-ef63f6b83cc3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.889193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2007adb709d24d7b8d7009d67d9dc9f0', 'request_id': 'req-2747c48b-28e9-442c-9249-ef63f6b83cc3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.899542 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] The token cache is being invalidate because project 2007adb709d24d7b8d7009d67d9dc9f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:38.899829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.901087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2747c48b-28e9-442c-9249-ef63f6b83cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:38.902592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1177/2345] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:38 2022] DELETE /identity/v3/projects/2007adb709d24d7b8d7009d67d9dc9f0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:38.907586 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddd55-2473-41e1-9e96-a9c0be720b1a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:38.907868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddd55-2473-41e1-9e96-a9c0be720b1a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:38.908140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddd55-2473-41e1-9e96-a9c0be720b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:38.926606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-07fddd55-2473-41e1-9e96-a9c0be720b1a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:38.927811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-07fddd55-2473-41e1-9e96-a9c0be720b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:38.986923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1169/2346] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:38.994414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37f3c8-d9e0-4e2e-abee-2723069801d6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:38.994700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37f3c8-d9e0-4e2e-abee-2723069801d6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:38.995040 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37f3c8-d9e0-4e2e-abee-2723069801d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.015172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9c37f3c8-d9e0-4e2e-abee-2723069801d6 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:39.016364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9c37f3c8-d9e0-4e2e-abee-2723069801d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:39.024329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1178/2347] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:39.029049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fddd55-2473-41e1-9e96-a9c0be720b1a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.038761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fddd55-2473-41e1-9e96-a9c0be720b1a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.043385 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da128bf-a480-4342-bd33-93b9e69ee5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.043982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da128bf-a480-4342-bd33-93b9e69ee5cc admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.044264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da128bf-a480-4342-bd33-93b9e69ee5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.044538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da128bf-a480-4342-bd33-93b9e69ee5cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.045187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da128bf-a480-4342-bd33-93b9e69ee5cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:05:39.045670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da128bf-a480-4342-bd33-93b9e69ee5cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.047198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da128bf-a480-4342-bd33-93b9e69ee5cc admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.060354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1170/2348] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:05:39.064794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37f3c8-d9e0-4e2e-abee-2723069801d6 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.074681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37f3c8-d9e0-4e2e-abee-2723069801d6 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.079569 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680a821-8762-4283-b716-19f594e31095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.080173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680a821-8762-4283-b716-19f594e31095 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.080553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680a821-8762-4283-b716-19f594e31095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.080830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680a821-8762-4283-b716-19f594e31095 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.081449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680a821-8762-4283-b716-19f594e31095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.083381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680a821-8762-4283-b716-19f594e31095 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.112044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1179/2349] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:05:39 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:39.116185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da128bf-a480-4342-bd33-93b9e69ee5cc admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.126562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da128bf-a480-4342-bd33-93b9e69ee5cc admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.166777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:39.189575 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc254d8a-399d-46db-874f-9ec221da97a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.190198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc254d8a-399d-46db-874f-9ec221da97a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.190460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc254d8a-399d-46db-874f-9ec221da97a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.190728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc254d8a-399d-46db-874f-9ec221da97a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.191386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc254d8a-399d-46db-874f-9ec221da97a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.193294 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc254d8a-399d-46db-874f-9ec221da97a9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.219123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1171/2350] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:05:39 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:05:39.220348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680a821-8762-4283-b716-19f594e31095 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.231786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680a821-8762-4283-b716-19f594e31095 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.236674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5193021-440f-416c-a21e-2565c6ed8711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.237438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5193021-440f-416c-a21e-2565c6ed8711 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.237668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5193021-440f-416c-a21e-2565c6ed8711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.237960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5193021-440f-416c-a21e-2565c6ed8711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.238870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5193021-440f-416c-a21e-2565c6ed8711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.240796 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5193021-440f-416c-a21e-2565c6ed8711 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.251690 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1180/2351] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:05:39.255761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc254d8a-399d-46db-874f-9ec221da97a9 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.266020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc254d8a-399d-46db-874f-9ec221da97a9 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.270813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc983edf-ae11-43a2-b03a-c0830e6b16ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.271453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc983edf-ae11-43a2-b03a-c0830e6b16ec admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.271754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc983edf-ae11-43a2-b03a-c0830e6b16ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.272050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc983edf-ae11-43a2-b03a-c0830e6b16ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.272684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc983edf-ae11-43a2-b03a-c0830e6b16ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.274299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc983edf-ae11-43a2-b03a-c0830e6b16ec admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.284974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1172/2352] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:39.288924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5193021-440f-416c-a21e-2565c6ed8711 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.299522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5193021-440f-416c-a21e-2565c6ed8711 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.304100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147c178-fb4d-4475-a1f6-bc99d844286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.304762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147c178-fb4d-4475-a1f6-bc99d844286e admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.305067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147c178-fb4d-4475-a1f6-bc99d844286e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.305346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147c178-fb4d-4475-a1f6-bc99d844286e admin admin] PATH_INFO: `/v3/projects/12278cea31e44a5fa9b0df03bdd6097b/users/7019d40625f84fd9973d2e2a2bc125fd/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.319898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147c178-fb4d-4475-a1f6-bc99d844286e admin admin] RBAC: Authorizing `identity:create_grant(project_id=12278cea31e44a5fa9b0df03bdd6097b, user_id=7019d40625f84fd9973d2e2a2bc125fd, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.322702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147c178-fb4d-4475-a1f6-bc99d844286e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.335382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1181/2353] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:39 2022] PUT /identity/v3/projects/12278cea31e44a5fa9b0df03bdd6097b/users/7019d40625f84fd9973d2e2a2bc125fd/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:05:39.346060 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc1fb3-7a27-47a2-b098-9f9b8b1f57df None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.346356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc1fb3-7a27-47a2-b098-9f9b8b1f57df None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.346676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc1fb3-7a27-47a2-b098-9f9b8b1f57df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.378542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a1cc1fb3-7a27-47a2-b098-9f9b8b1f57df None None] MFA Rules not processed for user `7019d40625f84fd9973d2e2a2bc125fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:39.379773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a1cc1fb3-7a27-47a2-b098-9f9b8b1f57df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:39.468255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1173/2354] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:39.469740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147c178-fb4d-4475-a1f6-bc99d844286e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.534787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147c178-fb4d-4475-a1f6-bc99d844286e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.539514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffc771-40b9-44a6-88f1-617c5e21b15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.540124 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffc771-40b9-44a6-88f1-617c5e21b15d admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.540405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffc771-40b9-44a6-88f1-617c5e21b15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.540703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffc771-40b9-44a6-88f1-617c5e21b15d admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.541332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdffc771-40b9-44a6-88f1-617c5e21b15d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.543280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdffc771-40b9-44a6-88f1-617c5e21b15d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.548137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:39.576980 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1182/2355] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:05:39 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:39.581003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc1fb3-7a27-47a2-b098-9f9b8b1f57df None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.591692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc1fb3-7a27-47a2-b098-9f9b8b1f57df None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.659626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d3a6c-9a21-4a57-9415-ce43d3c2423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.660232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d3a6c-9a21-4a57-9415-ce43d3c2423d admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.660512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d3a6c-9a21-4a57-9415-ce43d3c2423d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.660784 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d3a6c-9a21-4a57-9415-ce43d3c2423d admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.661400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d3a6c-9a21-4a57-9415-ce43d3c2423d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.663433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d3a6c-9a21-4a57-9415-ce43d3c2423d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.688406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1174/2356] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:05:39 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:39.689905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffc771-40b9-44a6-88f1-617c5e21b15d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.700215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffc771-40b9-44a6-88f1-617c5e21b15d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.704863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70566e7c-4d93-44e4-aa02-dd3dffb73d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.705525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70566e7c-4d93-44e4-aa02-dd3dffb73d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.705814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70566e7c-4d93-44e4-aa02-dd3dffb73d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.706088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70566e7c-4d93-44e4-aa02-dd3dffb73d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.706871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70566e7c-4d93-44e4-aa02-dd3dffb73d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.708587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70566e7c-4d93-44e4-aa02-dd3dffb73d64 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.719007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1183/2357] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:39.722945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d3a6c-9a21-4a57-9415-ce43d3c2423d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.733400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d3a6c-9a21-4a57-9415-ce43d3c2423d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.738159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939f0fb-6df1-4b1f-9749-618a959f09f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.738756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939f0fb-6df1-4b1f-9749-618a959f09f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.739073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939f0fb-6df1-4b1f-9749-618a959f09f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.739397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939f0fb-6df1-4b1f-9749-618a959f09f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.740077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939f0fb-6df1-4b1f-9749-618a959f09f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.741715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939f0fb-6df1-4b1f-9749-618a959f09f7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.752771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1175/2358] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:05:39.757174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70566e7c-4d93-44e4-aa02-dd3dffb73d64 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:39.766890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70566e7c-4d93-44e4-aa02-dd3dffb73d64 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:39.771655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641becfe-204e-46e9-8924-32ca400ceeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:39.772311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641becfe-204e-46e9-8924-32ca400ceeec admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.772583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641becfe-204e-46e9-8924-32ca400ceeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.772856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641becfe-204e-46e9-8924-32ca400ceeec admin admin] PATH_INFO: `/v3/projects/3672329a6a6b40baa332e866fc8f5eba/users/4a4613d315eb4c178f7581fbaf77ed6a/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.788808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641becfe-204e-46e9-8924-32ca400ceeec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3672329a6a6b40baa332e866fc8f5eba, user_id=4a4613d315eb4c178f7581fbaf77ed6a, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:39.792440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641becfe-204e-46e9-8924-32ca400ceeec admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:39.807201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1184/2359] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:39 2022] PUT /identity/v3/projects/3672329a6a6b40baa332e866fc8f5eba/users/4a4613d315eb4c178f7581fbaf77ed6a/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:39.818551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba055d-0a67-4ea2-864d-6c115df18bb5 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:39.818974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba055d-0a67-4ea2-864d-6c115df18bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:39.819271 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba055d-0a67-4ea2-864d-6c115df18bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:39.846722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0eba055d-0a67-4ea2-864d-6c115df18bb5 None None] MFA Rules not processed for user `4a4613d315eb4c178f7581fbaf77ed6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:39.847963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0eba055d-0a67-4ea2-864d-6c115df18bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:39.861571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:39.917538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1176/2360] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:39.922149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641becfe-204e-46e9-8924-32ca400ceeec admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:40.010936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641becfe-204e-46e9-8924-32ca400ceeec admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:40.015677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e25c7-2816-4bff-b173-da44f2e2632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:40.016277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e25c7-2816-4bff-b173-da44f2e2632d admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:40.016555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e25c7-2816-4bff-b173-da44f2e2632d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:40.016828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e25c7-2816-4bff-b173-da44f2e2632d admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:40.017461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e25c7-2816-4bff-b173-da44f2e2632d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:40.019388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e25c7-2816-4bff-b173-da44f2e2632d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:40.061768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1185/2361] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:05:39 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:05:40.065701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba055d-0a67-4ea2-864d-6c115df18bb5 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:40.077375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba055d-0a67-4ea2-864d-6c115df18bb5 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:40.135561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4a436-21f7-4676-9c5d-007f7a121cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:40.136174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4a436-21f7-4676-9c5d-007f7a121cad admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:40.136452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4a436-21f7-4676-9c5d-007f7a121cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:40.136722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4a436-21f7-4676-9c5d-007f7a121cad admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:40.137338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4a436-21f7-4676-9c5d-007f7a121cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:40.139269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4a436-21f7-4676-9c5d-007f7a121cad admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:40.163007 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1177/2362] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:05:40 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:05:40.167472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e25c7-2816-4bff-b173-da44f2e2632d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:40.177677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e25c7-2816-4bff-b173-da44f2e2632d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:40.183232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db28f-9e2e-4fb5-9419-9cd10d9fd668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:40.183873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db28f-9e2e-4fb5-9419-9cd10d9fd668 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:40.184156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db28f-9e2e-4fb5-9419-9cd10d9fd668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:40.184426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db28f-9e2e-4fb5-9419-9cd10d9fd668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:40.185077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408db28f-9e2e-4fb5-9419-9cd10d9fd668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:40.186676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408db28f-9e2e-4fb5-9419-9cd10d9fd668 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:40.201412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1186/2363] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:40.202244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4a436-21f7-4676-9c5d-007f7a121cad admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:40.214340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4a436-21f7-4676-9c5d-007f7a121cad admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:40.220604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75efd9-49b7-4b1c-b1eb-1e2f96395f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:40.221351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75efd9-49b7-4b1c-b1eb-1e2f96395f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:40.221706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75efd9-49b7-4b1c-b1eb-1e2f96395f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:40.222058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75efd9-49b7-4b1c-b1eb-1e2f96395f2d admin admin] PATH_INFO: `/v3/projects/5a4cd98207e94ead99b844ade62fd562/users/dff4035e50c04ce4b1a8f4f5b12584aa/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:40.240337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75efd9-49b7-4b1c-b1eb-1e2f96395f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4cd98207e94ead99b844ade62fd562, user_id=dff4035e50c04ce4b1a8f4f5b12584aa, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:40.244556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75efd9-49b7-4b1c-b1eb-1e2f96395f2d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:40.260889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1178/2364] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:40 2022] PUT /identity/v3/projects/5a4cd98207e94ead99b844ade62fd562/users/dff4035e50c04ce4b1a8f4f5b12584aa/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:40.264926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db28f-9e2e-4fb5-9419-9cd10d9fd668 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:40.275034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db28f-9e2e-4fb5-9419-9cd10d9fd668 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:40.331388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f6a90-0795-4549-9d69-583c78c1d64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:40.331913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f6a90-0795-4549-9d69-583c78c1d64d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:40.332191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f6a90-0795-4549-9d69-583c78c1d64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:40.332464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f6a90-0795-4549-9d69-583c78c1d64d admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:40.333093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f6a90-0795-4549-9d69-583c78c1d64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:40.334667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f6a90-0795-4549-9d69-583c78c1d64d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:40.344789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1187/2365] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:40.348908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75efd9-49b7-4b1c-b1eb-1e2f96395f2d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:40.358935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75efd9-49b7-4b1c-b1eb-1e2f96395f2d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:40.363461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe9bf8-beec-47d1-a0c0-dd597990a14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:40.364091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe9bf8-beec-47d1-a0c0-dd597990a14b admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:40.364364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe9bf8-beec-47d1-a0c0-dd597990a14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:40.364630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe9bf8-beec-47d1-a0c0-dd597990a14b admin admin] PATH_INFO: `/v3/projects/5a4cd98207e94ead99b844ade62fd562/users/dff4035e50c04ce4b1a8f4f5b12584aa/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:40.366512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe9bf8-beec-47d1-a0c0-dd597990a14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4cd98207e94ead99b844ade62fd562, user_id=dff4035e50c04ce4b1a8f4f5b12584aa, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:40.369172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe9bf8-beec-47d1-a0c0-dd597990a14b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:40.379221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1179/2366] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:40 2022] PUT /identity/v3/projects/5a4cd98207e94ead99b844ade62fd562/users/dff4035e50c04ce4b1a8f4f5b12584aa/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:40.389727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18894251-c47d-4024-8dfa-d2e49d1351c4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:40.390015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18894251-c47d-4024-8dfa-d2e49d1351c4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:40.390295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18894251-c47d-4024-8dfa-d2e49d1351c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:40.417877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-18894251-c47d-4024-8dfa-d2e49d1351c4 None None] MFA Rules not processed for user `dff4035e50c04ce4b1a8f4f5b12584aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:40.419104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-18894251-c47d-4024-8dfa-d2e49d1351c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:40.427018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:40.487024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1188/2367] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:40.498036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe9bf8-beec-47d1-a0c0-dd597990a14b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:40.499850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bbfe9bf8-beec-47d1-a0c0-dd597990a14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:40.557202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe9bf8-beec-47d1-a0c0-dd597990a14b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:40.560907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c1206-4a4e-4e4f-bc47-f45a0e207d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:40.561560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c1206-4a4e-4e4f-bc47-f45a0e207d5a service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:40.561840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c1206-4a4e-4e4f-bc47-f45a0e207d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:40.562110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c1206-4a4e-4e4f-bc47-f45a0e207d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:40.581778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:40.621531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c1206-4a4e-4e4f-bc47-f45a0e207d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:40.623151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c1206-4a4e-4e4f-bc47-f45a0e207d5a service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:40.632163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1180/2368] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:42.247454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18894251-c47d-4024-8dfa-d2e49d1351c4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:42.249559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-18894251-c47d-4024-8dfa-d2e49d1351c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:42.264164 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18894251-c47d-4024-8dfa-d2e49d1351c4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:42.273502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d052549-3729-457e-af56-4e0a24c89d4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:42.274751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d052549-3729-457e-af56-4e0a24c89d4a service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:42.274751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d052549-3729-457e-af56-4e0a24c89d4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:42.274751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d052549-3729-457e-af56-4e0a24c89d4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:42.376660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d052549-3729-457e-af56-4e0a24c89d4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:42.378744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d052549-3729-457e-af56-4e0a24c89d4a service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:42.396265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1189/2369] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:43.114977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c1206-4a4e-4e4f-bc47-f45a0e207d5a service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:43.124695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c1206-4a4e-4e4f-bc47-f45a0e207d5a service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:43.130046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59596a52-1962-44f0-b7e2-f7810c080212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:43.130807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59596a52-1962-44f0-b7e2-f7810c080212 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:43.131123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59596a52-1962-44f0-b7e2-f7810c080212 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:43.131401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59596a52-1962-44f0-b7e2-f7810c080212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:43.141100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59596a52-1962-44f0-b7e2-f7810c080212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:43.142483 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59596a52-1962-44f0-b7e2-f7810c080212 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:43.154223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1181/2370] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:49.406473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d052549-3729-457e-af56-4e0a24c89d4a service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:49.470266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d052549-3729-457e-af56-4e0a24c89d4a service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:49.474921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:49.475564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:49.475852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:49.476122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] PATH_INFO: `/v3/users/7019d40625f84fd9973d2e2a2bc125fd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:49.477245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7019d40625f84fd9973d2e2a2bc125fd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:49.479211 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:49.525092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:49.527477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7019d40625f84fd9973d2e2a2bc125fd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:49.535190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7019d40625f84fd9973d2e2a2bc125fd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:49.546766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7019d40625f84fd9973d2e2a2bc125fd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:49.547461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7019d40625f84fd9973d2e2a2bc125fd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:49.548059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7019d40625f84fd9973d2e2a2bc125fd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:49.560560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1190/2371] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:49 2022] DELETE /identity/v3/users/7019d40625f84fd9973d2e2a2bc125fd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:49.565048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6aad-a690-42fc-a123-73fe91cfe69a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:49.565598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6aad-a690-42fc-a123-73fe91cfe69a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:49.565598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6aad-a690-42fc-a123-73fe91cfe69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:49.584740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-840c6aad-a690-42fc-a123-73fe91cfe69a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:49.585829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-840c6aad-a690-42fc-a123-73fe91cfe69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:49.650674 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1182/2372] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:49.660228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:49.661604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:49.720087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc269a-514c-4850-944c-dda9a7265919 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:49.723900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724237-de2a-46b3-87af-d39d1b8c2f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:49.724628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724237-de2a-46b3-87af-d39d1b8c2f26 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:49.725005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724237-de2a-46b3-87af-d39d1b8c2f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:49.725377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724237-de2a-46b3-87af-d39d1b8c2f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:49.735801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724237-de2a-46b3-87af-d39d1b8c2f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:49.737303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724237-de2a-46b3-87af-d39d1b8c2f26 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:49.748895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1191/2373] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:50.045994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c6aad-a690-42fc-a123-73fe91cfe69a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:50.047498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-840c6aad-a690-42fc-a123-73fe91cfe69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:50.067936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c6aad-a690-42fc-a123-73fe91cfe69a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:50.075428 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:50.076164 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:50.076508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:50.076893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] PATH_INFO: `/v3/projects/12278cea31e44a5fa9b0df03bdd6097b` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:50.078263 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12278cea31e44a5fa9b0df03bdd6097b)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:50.082749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:50.144075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12278cea31e44a5fa9b0df03bdd6097b', 'request_id': 'req-25a355a1-3c15-452c-b6cf-f0088571a2d5'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.153868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12278cea31e44a5fa9b0df03bdd6097b', 'request_id': 'req-25a355a1-3c15-452c-b6cf-f0088571a2d5'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.162770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12278cea31e44a5fa9b0df03bdd6097b', 'request_id': 'req-25a355a1-3c15-452c-b6cf-f0088571a2d5'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.175393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] The token cache is being invalidate because project 12278cea31e44a5fa9b0df03bdd6097b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:50.175681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.176314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.180584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1183/2374] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:50 2022] DELETE /identity/v3/projects/12278cea31e44a5fa9b0df03bdd6097b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:50.181802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724237-de2a-46b3-87af-d39d1b8c2f26 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:50.191322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724237-de2a-46b3-87af-d39d1b8c2f26 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:50.249223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:50.249849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:50.250126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:50.250397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] PATH_INFO: `/v3/users/4a4613d315eb4c178f7581fbaf77ed6a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:50.251807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4613d315eb4c178f7581fbaf77ed6a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:50.253718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:50.302882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4613d315eb4c178f7581fbaf77ed6a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.311539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4613d315eb4c178f7581fbaf77ed6a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.319689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:50.325071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4613d315eb4c178f7581fbaf77ed6a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.325817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4613d315eb4c178f7581fbaf77ed6a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.326731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4613d315eb4c178f7581fbaf77ed6a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.340511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1192/2375] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:50 2022] DELETE /identity/v3/users/4a4613d315eb4c178f7581fbaf77ed6a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:50.670405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:50.672147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:50.687691 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a355a1-3c15-452c-b6cf-f0088571a2d5 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:50.719181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:50.748574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:50.749208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:50.749489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:50.749760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] PATH_INFO: `/v3/projects/3672329a6a6b40baa332e866fc8f5eba` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:50.751157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3672329a6a6b40baa332e866fc8f5eba)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:50.753075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:50.798353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3672329a6a6b40baa332e866fc8f5eba', 'request_id': 'req-a66d2d3b-1602-4460-a758-be2d0277e417'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.804282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3672329a6a6b40baa332e866fc8f5eba', 'request_id': 'req-a66d2d3b-1602-4460-a758-be2d0277e417'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.811342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3672329a6a6b40baa332e866fc8f5eba', 'request_id': 'req-a66d2d3b-1602-4460-a758-be2d0277e417'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.821182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] The token cache is being invalidate because project 3672329a6a6b40baa332e866fc8f5eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:50.821465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.822585 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.824061 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1184/2376] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:50 2022] DELETE /identity/v3/projects/3672329a6a6b40baa332e866fc8f5eba => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:50.828013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:50.829447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:50.842633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3f141-9a1a-4e3a-a437-7113576594b8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:50.897386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:50.897978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:50.898264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:50.898538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] PATH_INFO: `/v3/users/dff4035e50c04ce4b1a8f4f5b12584aa` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:50.899759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff4035e50c04ce4b1a8f4f5b12584aa)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:50.902275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:50.949997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff4035e50c04ce4b1a8f4f5b12584aa'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.957378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff4035e50c04ce4b1a8f4f5b12584aa'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.971230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff4035e50c04ce4b1a8f4f5b12584aa'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.971963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff4035e50c04ce4b1a8f4f5b12584aa'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.972569 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d8f8f74-3640-4e82-86c4-abd86cf9f112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff4035e50c04ce4b1a8f4f5b12584aa'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:50.986275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1193/2377] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:50 2022] DELETE /identity/v3/users/dff4035e50c04ce4b1a8f4f5b12584aa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:51.502709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:51.504220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:51.517347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d2d3b-1602-4460-a758-be2d0277e417 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:51.577170 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:51.577802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:51.578078 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:51.578348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] PATH_INFO: `/v3/projects/5a4cd98207e94ead99b844ade62fd562` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:51.579482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4cd98207e94ead99b844ade62fd562)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:51.581448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:51.627177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4cd98207e94ead99b844ade62fd562', 'request_id': 'req-335da44e-71b2-483a-8f3e-c9890df1ce75'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:51.633156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4cd98207e94ead99b844ade62fd562', 'request_id': 'req-335da44e-71b2-483a-8f3e-c9890df1ce75'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:51.640729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4cd98207e94ead99b844ade62fd562', 'request_id': 'req-335da44e-71b2-483a-8f3e-c9890df1ce75'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:51.644647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:51.653348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] The token cache is being invalidate because project 5a4cd98207e94ead99b844ade62fd562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:51.653610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:51.654324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-335da44e-71b2-483a-8f3e-c9890df1ce75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:51.656230 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1185/2378] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:51 2022] DELETE /identity/v3/projects/5a4cd98207e94ead99b844ade62fd562 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:51.661147 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c15eb-6078-4775-9977-5290a246a3fe None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:51.661439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c15eb-6078-4775-9977-5290a246a3fe None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:51.661719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c15eb-6078-4775-9977-5290a246a3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:51.683111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9e8c15eb-6078-4775-9977-5290a246a3fe None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:51.684232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9e8c15eb-6078-4775-9977-5290a246a3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:51.741578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1194/2379] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:51.751418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b8b2c-d25a-40b6-a8ce-d29c5df05664 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:51.751877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b8b2c-d25a-40b6-a8ce-d29c5df05664 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:51.751968 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b8b2c-d25a-40b6-a8ce-d29c5df05664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:51.772105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d31b8b2c-d25a-40b6-a8ce-d29c5df05664 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:51.773283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d31b8b2c-d25a-40b6-a8ce-d29c5df05664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:51.781403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1186/2380] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:51.785565 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c15eb-6078-4775-9977-5290a246a3fe None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:51.792082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:51.795896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c15eb-6078-4775-9977-5290a246a3fe None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:51.801742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8e22a-4ba5-4ccb-8849-58eb372cdf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:51.802353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8e22a-4ba5-4ccb-8849-58eb372cdf84 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:51.802832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8e22a-4ba5-4ccb-8849-58eb372cdf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:51.803308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8e22a-4ba5-4ccb-8849-58eb372cdf84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:51.804103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8e22a-4ba5-4ccb-8849-58eb372cdf84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:05:51.804556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8e22a-4ba5-4ccb-8849-58eb372cdf84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:51.806167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8e22a-4ba5-4ccb-8849-58eb372cdf84 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:51.818120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1195/2381] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:05:51.822290 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b8b2c-d25a-40b6-a8ce-d29c5df05664 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:51.831444 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b8b2c-d25a-40b6-a8ce-d29c5df05664 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:51.836038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb46827-a79f-4ed2-81ef-4a8d0608a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:51.836628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb46827-a79f-4ed2-81ef-4a8d0608a2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:51.836903 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb46827-a79f-4ed2-81ef-4a8d0608a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:51.837173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb46827-a79f-4ed2-81ef-4a8d0608a2a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:51.837798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb46827-a79f-4ed2-81ef-4a8d0608a2a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:51.839719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb46827-a79f-4ed2-81ef-4a8d0608a2a8 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:51.868323 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1187/2382] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:05:51 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:51.873022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8e22a-4ba5-4ccb-8849-58eb372cdf84 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:51.883193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8e22a-4ba5-4ccb-8849-58eb372cdf84 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:51.938193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bad49-69b0-429c-85c4-f618c7f6f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:51.938710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bad49-69b0-429c-85c4-f618c7f6f53d admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:51.939008 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bad49-69b0-429c-85c4-f618c7f6f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:51.939280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bad49-69b0-429c-85c4-f618c7f6f53d admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:51.939888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bad49-69b0-429c-85c4-f618c7f6f53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:51.941775 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bad49-69b0-429c-85c4-f618c7f6f53d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:51.962196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1196/2383] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:05:51 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:51.966826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb46827-a79f-4ed2-81ef-4a8d0608a2a8 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:51.977256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb46827-a79f-4ed2-81ef-4a8d0608a2a8 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:51.982433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46223c-3bff-4d95-b1c1-a1ae4e7dd890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:51.983053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46223c-3bff-4d95-b1c1-a1ae4e7dd890 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:51.983334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46223c-3bff-4d95-b1c1-a1ae4e7dd890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:51.983607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46223c-3bff-4d95-b1c1-a1ae4e7dd890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:51.984247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46223c-3bff-4d95-b1c1-a1ae4e7dd890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:51.985829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46223c-3bff-4d95-b1c1-a1ae4e7dd890 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:51.995738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1188/2384] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:52.001953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bad49-69b0-429c-85c4-f618c7f6f53d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:52.012489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bad49-69b0-429c-85c4-f618c7f6f53d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:52.017044 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613400-56f6-4eb1-9180-5893fee8185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:52.017631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613400-56f6-4eb1-9180-5893fee8185a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:52.017906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613400-56f6-4eb1-9180-5893fee8185a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:52.018172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613400-56f6-4eb1-9180-5893fee8185a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:52.018804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c613400-56f6-4eb1-9180-5893fee8185a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:52.020406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c613400-56f6-4eb1-9180-5893fee8185a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:52.031146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1197/2385] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:52.035131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46223c-3bff-4d95-b1c1-a1ae4e7dd890 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:52.045384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46223c-3bff-4d95-b1c1-a1ae4e7dd890 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:52.050038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c568235-ec06-4e5f-aaad-191f92a9dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:52.050671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c568235-ec06-4e5f-aaad-191f92a9dcf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:52.050995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c568235-ec06-4e5f-aaad-191f92a9dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:52.051251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c568235-ec06-4e5f-aaad-191f92a9dcf6 admin admin] PATH_INFO: `/v3/projects/8eb9f56c8cb74e0b9210c4e104cde2fc/users/ffeb32ae7146432bbefdc4889c7a11a0/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:52.065998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c568235-ec06-4e5f-aaad-191f92a9dcf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb9f56c8cb74e0b9210c4e104cde2fc, user_id=ffeb32ae7146432bbefdc4889c7a11a0, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:52.068816 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c568235-ec06-4e5f-aaad-191f92a9dcf6 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:52.081838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1189/2386] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:52 2022] PUT /identity/v3/projects/8eb9f56c8cb74e0b9210c4e104cde2fc/users/ffeb32ae7146432bbefdc4889c7a11a0/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:52.092204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb06b4-06a4-479f-8460-1f690d3fba70 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:52.092513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb06b4-06a4-479f-8460-1f690d3fba70 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:52.092787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb06b4-06a4-479f-8460-1f690d3fba70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:52.120735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-89bb06b4-06a4-479f-8460-1f690d3fba70 None None] MFA Rules not processed for user `ffeb32ae7146432bbefdc4889c7a11a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:52.121948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-89bb06b4-06a4-479f-8460-1f690d3fba70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:52.183743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1198/2387] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:52.194873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c568235-ec06-4e5f-aaad-191f92a9dcf6 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:52.196579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8c568235-ec06-4e5f-aaad-191f92a9dcf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:52.254368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c568235-ec06-4e5f-aaad-191f92a9dcf6 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:52.258011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50327119-5e58-4403-86f6-e4453e902196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:52.258605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50327119-5e58-4403-86f6-e4453e902196 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:52.258883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50327119-5e58-4403-86f6-e4453e902196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:52.259185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50327119-5e58-4403-86f6-e4453e902196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:52.267803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50327119-5e58-4403-86f6-e4453e902196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:52.269217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50327119-5e58-4403-86f6-e4453e902196 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:52.277713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1190/2388] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:52.302084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb06b4-06a4-479f-8460-1f690d3fba70 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:52.362933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb06b4-06a4-479f-8460-1f690d3fba70 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:52.367533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:52.368150 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:52.368431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:52.368707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] PATH_INFO: `/v3/users/ffeb32ae7146432bbefdc4889c7a11a0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:52.370374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffeb32ae7146432bbefdc4889c7a11a0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:52.372324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:52.373778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:52.422681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffeb32ae7146432bbefdc4889c7a11a0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:52.430013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffeb32ae7146432bbefdc4889c7a11a0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:52.442334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffeb32ae7146432bbefdc4889c7a11a0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:52.442989 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffeb32ae7146432bbefdc4889c7a11a0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:52.443613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffeb32ae7146432bbefdc4889c7a11a0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:52.460281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1199/2389] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:05:52 2022] DELETE /identity/v3/users/ffeb32ae7146432bbefdc4889c7a11a0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:52.465317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03722939-e9fa-4245-9cc0-c58305a86b15 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:52.465600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03722939-e9fa-4245-9cc0-c58305a86b15 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:52.465898 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03722939-e9fa-4245-9cc0-c58305a86b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:52.484406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-03722939-e9fa-4245-9cc0-c58305a86b15 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:52.485571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-03722939-e9fa-4245-9cc0-c58305a86b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:52.544935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1191/2390] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:52.554142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:52.555579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:52.612835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a2a64-ee4d-4c91-aba7-1900e2984b1b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:52.616630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec930e-99dd-4003-8268-d4d921d84d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:52.617242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec930e-99dd-4003-8268-d4d921d84d9a service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:52.617519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec930e-99dd-4003-8268-d4d921d84d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:52.617796 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec930e-99dd-4003-8268-d4d921d84d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:52.628697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec930e-99dd-4003-8268-d4d921d84d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:52.630240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec930e-99dd-4003-8268-d4d921d84d9a service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:52.640682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1200/2391] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:52.913094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03722939-e9fa-4245-9cc0-c58305a86b15 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:52.914498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-03722939-e9fa-4245-9cc0-c58305a86b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:52.926114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03722939-e9fa-4245-9cc0-c58305a86b15 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:52.930758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:52.931969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:52.932327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:52.932604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] PATH_INFO: `/v3/projects/8eb9f56c8cb74e0b9210c4e104cde2fc` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:52.934064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb9f56c8cb74e0b9210c4e104cde2fc)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:52.936015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:52.969643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:52.985632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb9f56c8cb74e0b9210c4e104cde2fc', 'request_id': 'req-186b8ee3-c21a-49a5-9f13-c7a152c426d1'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:52.991618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb9f56c8cb74e0b9210c4e104cde2fc', 'request_id': 'req-186b8ee3-c21a-49a5-9f13-c7a152c426d1'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:53.001001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb9f56c8cb74e0b9210c4e104cde2fc', 'request_id': 'req-186b8ee3-c21a-49a5-9f13-c7a152c426d1'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:53.011828 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] The token cache is being invalidate because project 8eb9f56c8cb74e0b9210c4e104cde2fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:05:53.012110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:53.012721 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-186b8ee3-c21a-49a5-9f13-c7a152c426d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:05:53.014216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1192/2392] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:05:52 2022] DELETE /identity/v3/projects/8eb9f56c8cb74e0b9210c4e104cde2fc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:53.019194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953dfad-fd0b-4b12-a487-277e81868094 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.019639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953dfad-fd0b-4b12-a487-277e81868094 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.019756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953dfad-fd0b-4b12-a487-277e81868094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.038859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a953dfad-fd0b-4b12-a487-277e81868094 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:53.040080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a953dfad-fd0b-4b12-a487-277e81868094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:53.098849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1201/2393] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:53.107353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01db40b-bd0d-4260-bf90-4acd4c01f3d2 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.107686 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01db40b-bd0d-4260-bf90-4acd4c01f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.107964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01db40b-bd0d-4260-bf90-4acd4c01f3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.127871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e01db40b-bd0d-4260-bf90-4acd4c01f3d2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:53.129036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e01db40b-bd0d-4260-bf90-4acd4c01f3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:53.137268 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1193/2394] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:53.141352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953dfad-fd0b-4b12-a487-277e81868094 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:53.151430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953dfad-fd0b-4b12-a487-277e81868094 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:53.155979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912a6ef-9d6a-477e-a4f3-681326da3fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:53.156573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912a6ef-9d6a-477e-a4f3-681326da3fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.156852 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912a6ef-9d6a-477e-a4f3-681326da3fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.157123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912a6ef-9d6a-477e-a4f3-681326da3fc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.157762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912a6ef-9d6a-477e-a4f3-681326da3fc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:05:53.158062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912a6ef-9d6a-477e-a4f3-681326da3fc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:53.159686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912a6ef-9d6a-477e-a4f3-681326da3fc9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:53.170851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1202/2395] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:05:53.174843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01db40b-bd0d-4260-bf90-4acd4c01f3d2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:53.183956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01db40b-bd0d-4260-bf90-4acd4c01f3d2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:53.188521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6fb7c-6a70-4471-807c-a751728dd589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:53.189108 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6fb7c-6a70-4471-807c-a751728dd589 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.189384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6fb7c-6a70-4471-807c-a751728dd589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.189649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6fb7c-6a70-4471-807c-a751728dd589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.190262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6fb7c-6a70-4471-807c-a751728dd589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:53.192204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6fb7c-6a70-4471-807c-a751728dd589 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:53.219251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1194/2396] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:05:53 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:53.223188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912a6ef-9d6a-477e-a4f3-681326da3fc9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:53.232882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912a6ef-9d6a-477e-a4f3-681326da3fc9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:53.237502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:53.290334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4ba2f-56ee-40bf-a085-ff76d6cd914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:53.290927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4ba2f-56ee-40bf-a085-ff76d6cd914f admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.291234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4ba2f-56ee-40bf-a085-ff76d6cd914f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.291504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4ba2f-56ee-40bf-a085-ff76d6cd914f admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.292120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4ba2f-56ee-40bf-a085-ff76d6cd914f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:53.294018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4ba2f-56ee-40bf-a085-ff76d6cd914f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:53.314352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1203/2397] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:05:53 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:53.318448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6fb7c-6a70-4471-807c-a751728dd589 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:53.328975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6fb7c-6a70-4471-807c-a751728dd589 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:53.333467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df83faa-73ab-4f95-a381-a6ac9664e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:53.334045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df83faa-73ab-4f95-a381-a6ac9664e2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.334319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df83faa-73ab-4f95-a381-a6ac9664e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.334584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df83faa-73ab-4f95-a381-a6ac9664e2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.335246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df83faa-73ab-4f95-a381-a6ac9664e2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:53.336842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df83faa-73ab-4f95-a381-a6ac9664e2b8 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:53.347051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1195/2398] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:53.350625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4ba2f-56ee-40bf-a085-ff76d6cd914f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:53.360308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4ba2f-56ee-40bf-a085-ff76d6cd914f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:53.364876 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9c68-0f58-42c5-8b44-99b2f255a180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:53.365457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9c68-0f58-42c5-8b44-99b2f255a180 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.365738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9c68-0f58-42c5-8b44-99b2f255a180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.366011 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9c68-0f58-42c5-8b44-99b2f255a180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.366642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e9c68-0f58-42c5-8b44-99b2f255a180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:53.368251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e9c68-0f58-42c5-8b44-99b2f255a180 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:53.379336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1204/2399] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:53.383461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df83faa-73ab-4f95-a381-a6ac9664e2b8 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:53.393669 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df83faa-73ab-4f95-a381-a6ac9664e2b8 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:53.398594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b9de8-eece-4264-b19b-c50353463549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:53.399421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b9de8-eece-4264-b19b-c50353463549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.399770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b9de8-eece-4264-b19b-c50353463549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.400217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b9de8-eece-4264-b19b-c50353463549 admin admin] PATH_INFO: `/v3/projects/aa338ea60ddf4f53a71c19d51b21edb8/users/7a9c654fc9bc4ad4b530d3f4df5c24c8/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.416146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b9de8-eece-4264-b19b-c50353463549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa338ea60ddf4f53a71c19d51b21edb8, user_id=7a9c654fc9bc4ad4b530d3f4df5c24c8, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:53.418970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b9de8-eece-4264-b19b-c50353463549 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:53.432504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1196/2400] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:53 2022] PUT /identity/v3/projects/aa338ea60ddf4f53a71c19d51b21edb8/users/7a9c654fc9bc4ad4b530d3f4df5c24c8/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:53.437786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2d270-9884-4051-819c-80723032419a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.438072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2d270-9884-4051-819c-80723032419a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.438370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2d270-9884-4051-819c-80723032419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.457880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-99f2d270-9884-4051-819c-80723032419a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:53.459095 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-99f2d270-9884-4051-819c-80723032419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:53.646188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1205/2401] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:53.656670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b9de8-eece-4264-b19b-c50353463549 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:53.658402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ba4b9de8-eece-4264-b19b-c50353463549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:53.719521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b9de8-eece-4264-b19b-c50353463549 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:53.723364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1f354-eb2c-4275-9fbe-1ea46995c7f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:53.723990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f354-eb2c-4275-9fbe-1ea46995c7f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.724302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f354-eb2c-4275-9fbe-1ea46995c7f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.724598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f354-eb2c-4275-9fbe-1ea46995c7f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.726228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef309641-b16d-4b68-9b6b-9d828df8f13d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:53.726511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef309641-b16d-4b68-9b6b-9d828df8f13d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:53.726787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef309641-b16d-4b68-9b6b-9d828df8f13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:53.734753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1f354-eb2c-4275-9fbe-1ea46995c7f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:53.736328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1f354-eb2c-4275-9fbe-1ea46995c7f7 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:53.746383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1197/2402] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:53.746854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ef309641-b16d-4b68-9b6b-9d828df8f13d None None] MFA Rules not processed for user `55ae4dbafed74e9dbeab9538898cf1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:53.748126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ef309641-b16d-4b68-9b6b-9d828df8f13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:53.807821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1206/2403] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:56.324184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ebf39-8895-4322-ad99-09b05cfe1816 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:56.324708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ebf39-8895-4322-ad99-09b05cfe1816 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:56.325299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ebf39-8895-4322-ad99-09b05cfe1816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:56.384233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-cf0ebf39-8895-4322-ad99-09b05cfe1816 None None] MFA Rules not processed for user `befbca3738f0481296b63c70d6d2a797`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:56.385391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-cf0ebf39-8895-4322-ad99-09b05cfe1816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:56.459201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:56.471307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1198/2404] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:58.645629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef309641-b16d-4b68-9b6b-9d828df8f13d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:58.647241 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ef309641-b16d-4b68-9b6b-9d828df8f13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:58.669821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef309641-b16d-4b68-9b6b-9d828df8f13d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:58.689428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdabe29-f19a-4c51-950a-9b9e6bb136b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:58.690046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdabe29-f19a-4c51-950a-9b9e6bb136b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:58.695366 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdabe29-f19a-4c51-950a-9b9e6bb136b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:58.695725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdabe29-f19a-4c51-950a-9b9e6bb136b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:58.702743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9fdabe29-f19a-4c51-950a-9b9e6bb136b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:58.787013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:05:58.853993 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdabe29-f19a-4c51-950a-9b9e6bb136b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:58.856105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdabe29-f19a-4c51-950a-9b9e6bb136b5 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:58.881425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1207/2405] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:05:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:05:59.172434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8aa23-94f0-4f60-9b7e-c262bfeccf29 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:59.172929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8aa23-94f0-4f60-9b7e-c262bfeccf29 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:59.173014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8aa23-94f0-4f60-9b7e-c262bfeccf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:59.244580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-83d8aa23-94f0-4f60-9b7e-c262bfeccf29 None None] MFA Rules not processed for user `7a9c654fc9bc4ad4b530d3f4df5c24c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:05:59.246328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-83d8aa23-94f0-4f60-9b7e-c262bfeccf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:05:59.376113 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1199/2406] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:05:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:05:59.380674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdabe29-f19a-4c51-950a-9b9e6bb136b5 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:59.394740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdabe29-f19a-4c51-950a-9b9e6bb136b5 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:59.433821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c24cd9-de15-4870-95c4-874bcb117f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:59.434446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c24cd9-de15-4870-95c4-874bcb117f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:59.434778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c24cd9-de15-4870-95c4-874bcb117f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:59.435248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c24cd9-de15-4870-95c4-874bcb117f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:59.436275 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c24cd9-de15-4870-95c4-874bcb117f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:59.438450 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c24cd9-de15-4870-95c4-874bcb117f3e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:59.514252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1208/2407] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:05:59 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:59.516355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8aa23-94f0-4f60-9b7e-c262bfeccf29 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:59.535691 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8aa23-94f0-4f60-9b7e-c262bfeccf29 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:59.640894 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbae11e-8d52-4d45-82db-06cf86a8171d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:59.641493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbae11e-8d52-4d45-82db-06cf86a8171d admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:59.641871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbae11e-8d52-4d45-82db-06cf86a8171d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:59.642204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbae11e-8d52-4d45-82db-06cf86a8171d admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:59.642909 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbae11e-8d52-4d45-82db-06cf86a8171d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:59.645757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbae11e-8d52-4d45-82db-06cf86a8171d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:59.678446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1200/2408] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:05:59 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:05:59.679832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c24cd9-de15-4870-95c4-874bcb117f3e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:59.692291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c24cd9-de15-4870-95c4-874bcb117f3e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:59.697704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62c77c-9829-4eda-9a0c-e900f47a7eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:59.698567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62c77c-9829-4eda-9a0c-e900f47a7eab admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:59.699077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62c77c-9829-4eda-9a0c-e900f47a7eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:59.699415 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62c77c-9829-4eda-9a0c-e900f47a7eab admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:59.700393 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62c77c-9829-4eda-9a0c-e900f47a7eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:59.701853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62c77c-9829-4eda-9a0c-e900f47a7eab admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:59.714583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1209/2409] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:59.718714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbae11e-8d52-4d45-82db-06cf86a8171d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:59.729574 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbae11e-8d52-4d45-82db-06cf86a8171d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:59.736224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7ff08-12e1-4248-83f2-e44bd0ebbf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:59.737034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7ff08-12e1-4248-83f2-e44bd0ebbf53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:59.737380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7ff08-12e1-4248-83f2-e44bd0ebbf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:59.737722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7ff08-12e1-4248-83f2-e44bd0ebbf53 admin admin] PATH_INFO: `/v3/projects/d56982c987cb47e39c611ea9dddbc6e2/users/61eb6f0791b9440fa355609f4c8fb532/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:59.760203 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7ff08-12e1-4248-83f2-e44bd0ebbf53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56982c987cb47e39c611ea9dddbc6e2, user_id=61eb6f0791b9440fa355609f4c8fb532, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:59.763295 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7ff08-12e1-4248-83f2-e44bd0ebbf53 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:59.782453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1201/2410] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:59 2022] PUT /identity/v3/projects/d56982c987cb47e39c611ea9dddbc6e2/users/61eb6f0791b9440fa355609f4c8fb532/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:05:59.787604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62c77c-9829-4eda-9a0c-e900f47a7eab admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:59.801273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62c77c-9829-4eda-9a0c-e900f47a7eab admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:59.883239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51014-2bff-4cca-8393-0cf4f2bb557c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:59.883973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51014-2bff-4cca-8393-0cf4f2bb557c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:59.884559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51014-2bff-4cca-8393-0cf4f2bb557c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:59.884956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51014-2bff-4cca-8393-0cf4f2bb557c admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:59.885716 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51014-2bff-4cca-8393-0cf4f2bb557c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:59.887485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51014-2bff-4cca-8393-0cf4f2bb557c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:59.906262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7ff08-12e1-4248-83f2-e44bd0ebbf53 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:05:59.906602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1210/2411] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:05:59.919678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7ff08-12e1-4248-83f2-e44bd0ebbf53 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:05:59.936512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc0224-b963-46d2-a7e4-bfd9d40d64ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:05:59.937386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc0224-b963-46d2-a7e4-bfd9d40d64ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:05:59.937747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc0224-b963-46d2-a7e4-bfd9d40d64ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:05:59.938094 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc0224-b963-46d2-a7e4-bfd9d40d64ac admin admin] PATH_INFO: `/v3/projects/d56982c987cb47e39c611ea9dddbc6e2/users/61eb6f0791b9440fa355609f4c8fb532/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:05:59.941077 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc0224-b963-46d2-a7e4-bfd9d40d64ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56982c987cb47e39c611ea9dddbc6e2, user_id=61eb6f0791b9440fa355609f4c8fb532, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:05:59.946073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc0224-b963-46d2-a7e4-bfd9d40d64ac admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:05:59.955918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1202/2412] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:05:59 2022] PUT /identity/v3/projects/d56982c987cb47e39c611ea9dddbc6e2/users/61eb6f0791b9440fa355609f4c8fb532/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:06:07.957962 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99888396-21e6-4a3a-9eb9-e1de147eecf2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:06:07.959692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99888396-21e6-4a3a-9eb9-e1de147eecf2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:06:07.960182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99888396-21e6-4a3a-9eb9-e1de147eecf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:06:08.029373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-99888396-21e6-4a3a-9eb9-e1de147eecf2 None None] MFA Rules not processed for user `61eb6f0791b9440fa355609f4c8fb532`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:06:08.031444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-99888396-21e6-4a3a-9eb9-e1de147eecf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:06:08.294703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1211/2413] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:06:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:06:08.307549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc0224-b963-46d2-a7e4-bfd9d40d64ac admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:06:08.309910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b1bc0224-b963-46d2-a7e4-bfd9d40d64ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:06:08.399033 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:06:08.441172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc0224-b963-46d2-a7e4-bfd9d40d64ac admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:06:08.447947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b7357-8947-433b-91f0-24279e2ae240 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:06:08.448802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b7357-8947-433b-91f0-24279e2ae240 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:06:08.449253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b7357-8947-433b-91f0-24279e2ae240 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:06:08.449531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b7357-8947-433b-91f0-24279e2ae240 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:06:08.530467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b7357-8947-433b-91f0-24279e2ae240 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:06:08.532740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b7357-8947-433b-91f0-24279e2ae240 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:06:08.543492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1203/2414] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:06:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:06:15.209080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99888396-21e6-4a3a-9eb9-e1de147eecf2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:06:15.210635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-99888396-21e6-4a3a-9eb9-e1de147eecf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:06:15.231296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:06:15.292873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99888396-21e6-4a3a-9eb9-e1de147eecf2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:06:15.298412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286206d1-2dcd-4a7c-96d5-1843abcb542c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:06:15.299075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286206d1-2dcd-4a7c-96d5-1843abcb542c service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:06:15.299357 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286206d1-2dcd-4a7c-96d5-1843abcb542c service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:06:15.299631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286206d1-2dcd-4a7c-96d5-1843abcb542c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:06:15.301405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-286206d1-2dcd-4a7c-96d5-1843abcb542c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:06:15.314487 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286206d1-2dcd-4a7c-96d5-1843abcb542c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:06:15.316120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286206d1-2dcd-4a7c-96d5-1843abcb542c service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:06:15.327564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1212/2415] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:06:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:06:18.539011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=37 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:06:25.322931 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=25 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:06:30.301715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029628c-497d-47dc-a003-5e24a8408b67 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:06:30.302267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029628c-497d-47dc-a003-5e24a8408b67 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:06:30.302267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029628c-497d-47dc-a003-5e24a8408b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:06:30.321658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-1029628c-497d-47dc-a003-5e24a8408b67 None None] MFA Rules not processed for user `7a9c654fc9bc4ad4b530d3f4df5c24c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:06:30.322838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1029628c-497d-47dc-a003-5e24a8408b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:06:30.329563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1204/2416] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:06:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:06:40.315886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:06:55.225916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286206d1-2dcd-4a7c-96d5-1843abcb542c service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:06:55.290158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286206d1-2dcd-4a7c-96d5-1843abcb542c service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:06:55.293755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13babbee-c674-4b7f-ab00-c5249144ef15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:06:55.294356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13babbee-c674-4b7f-ab00-c5249144ef15 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:06:55.294632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13babbee-c674-4b7f-ab00-c5249144ef15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:06:55.294898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13babbee-c674-4b7f-ab00-c5249144ef15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:06:55.303905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13babbee-c674-4b7f-ab00-c5249144ef15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:06:55.305883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13babbee-c674-4b7f-ab00-c5249144ef15 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:06:55.315875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1213/2417] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:06:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:06:57.817555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e9f18-a24f-4e23-b517-2c720a8e1071 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:06:57.818455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e9f18-a24f-4e23-b517-2c720a8e1071 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:06:57.818876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e9f18-a24f-4e23-b517-2c720a8e1071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:06:57.848539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-532e9f18-a24f-4e23-b517-2c720a8e1071 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:06:57.849733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-532e9f18-a24f-4e23-b517-2c720a8e1071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:06:57.944029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1205/2418] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:06:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:06:58.006470 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13babbee-c674-4b7f-ab00-c5249144ef15 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:06:58.009300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-13babbee-c674-4b7f-ab00-c5249144ef15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:06:58.189895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13babbee-c674-4b7f-ab00-c5249144ef15 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:06:58.196884 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04858e94-f682-4b9e-9159-8c60d83e22d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:06:58.197774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04858e94-f682-4b9e-9159-8c60d83e22d9 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:06:58.198271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04858e94-f682-4b9e-9159-8c60d83e22d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:06:58.198746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04858e94-f682-4b9e-9159-8c60d83e22d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:06:58.208439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04858e94-f682-4b9e-9159-8c60d83e22d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:06:58.211106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04858e94-f682-4b9e-9159-8c60d83e22d9 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:06:58.221754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1214/2419] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:06:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:07:03.891542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e9f18-a24f-4e23-b517-2c720a8e1071 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:07:03.893590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-532e9f18-a24f-4e23-b517-2c720a8e1071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:04.051618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e9f18-a24f-4e23-b517-2c720a8e1071 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:07:04.064456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:07:04.065702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:04.066191 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:04.066820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] PATH_INFO: `/v3/users/55ae4dbafed74e9dbeab9538898cf1f6` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:04.068305 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ae4dbafed74e9dbeab9538898cf1f6)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:07:04.072168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:07:04.167011 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ae4dbafed74e9dbeab9538898cf1f6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.186411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ae4dbafed74e9dbeab9538898cf1f6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.201630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ae4dbafed74e9dbeab9538898cf1f6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.227209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ae4dbafed74e9dbeab9538898cf1f6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.228084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-29afd53a-275f-4d48-bd9e-95401f7edc83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ae4dbafed74e9dbeab9538898cf1f6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.235729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1206/2420] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:07:03 2022] DELETE /identity/v3/users/55ae4dbafed74e9dbeab9538898cf1f6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:07:04.530223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04858e94-f682-4b9e-9159-8c60d83e22d9 service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:07:04.543992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04858e94-f682-4b9e-9159-8c60d83e22d9 service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:07:04.672689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:07:04.673846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:04.674129 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:04.674466 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] PATH_INFO: `/v3/projects/0c42b50ff98c40a6b2f47f289538a330` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:04.680305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c42b50ff98c40a6b2f47f289538a330)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:07:04.680305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:07:04.728809 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:07:04.748918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c42b50ff98c40a6b2f47f289538a330', 'request_id': 'req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.758484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c42b50ff98c40a6b2f47f289538a330', 'request_id': 'req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.772647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c42b50ff98c40a6b2f47f289538a330', 'request_id': 'req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.789856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] The token cache is being invalidate because project 0c42b50ff98c40a6b2f47f289538a330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:07:04.790515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.791495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fd0bc667-7f3e-4dcf-8cc9-44942f99f69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:07:04.796977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1215/2421] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:07:04 2022] DELETE /identity/v3/projects/0c42b50ff98c40a6b2f47f289538a330 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:07:06.983678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6342d-062f-464c-b855-de476aa58a73 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:06.984688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6342d-062f-464c-b855-de476aa58a73 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:06.985223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6342d-062f-464c-b855-de476aa58a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:07.007833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-13d6342d-062f-464c-b855-de476aa58a73 None None] MFA Rules not processed for user `2dd3653b931643318f1a1b0820620277`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:07:07.009380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-13d6342d-062f-464c-b855-de476aa58a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:07.033031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:07:07.083044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1207/2422] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:07:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:07:14.785682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:07:14.900218 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433aaeb-d3ee-47bd-8c16-d2ca91d63389 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:14.900886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433aaeb-d3ee-47bd-8c16-d2ca91d63389 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:14.901228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433aaeb-d3ee-47bd-8c16-d2ca91d63389 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:14.903104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1216/2423] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 10:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:07:14.909493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923adcc-4698-47ae-9a25-48ba165cee5a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:14.910170 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923adcc-4698-47ae-9a25-48ba165cee5a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:14.910687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923adcc-4698-47ae-9a25-48ba165cee5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:14.934933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-3923adcc-4698-47ae-9a25-48ba165cee5a None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:07:14.937592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3923adcc-4698-47ae-9a25-48ba165cee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:15.031644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1208/2424] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 10:07:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:07:15.045009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433aaeb-d3ee-47bd-8c16-d2ca91d63389 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:07:15.046891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9433aaeb-d3ee-47bd-8c16-d2ca91d63389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:15.109353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433aaeb-d3ee-47bd-8c16-d2ca91d63389 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:07:15.112881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6db480-34b0-4a20-a25e-6ad1135816b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8f456f0d15b4d068cfcd6476563247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:07:15.113518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6db480-34b0-4a20-a25e-6ad1135816b9 service glance] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:15.113797 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6db480-34b0-4a20-a25e-6ad1135816b9 service glance] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:15.114070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6db480-34b0-4a20-a25e-6ad1135816b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:15.124404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6db480-34b0-4a20-a25e-6ad1135816b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:07:15.126019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6db480-34b0-4a20-a25e-6ad1135816b9 service glance] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:07:15.137045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1217/2425] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:07:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:07:15.216963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93940c-9176-477b-b22c-bc09fae40d34 None None] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:15.217293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93940c-9176-477b-b22c-bc09fae40d34 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:15.217666 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93940c-9176-477b-b22c-bc09fae40d34 None None] PATH_INFO: `/` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:15.219317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1209/2426] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 10:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:07:15.224249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c21439-ba53-43ea-9332-6aef193a90d0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:15.224536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c21439-ba53-43ea-9332-6aef193a90d0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:15.224850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c21439-ba53-43ea-9332-6aef193a90d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:15.244900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c0c21439-ba53-43ea-9332-6aef193a90d0 None None] MFA Rules not processed for user `ab9b7576a3834862af6abb7e0416f158`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:07:15.246099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c0c21439-ba53-43ea-9332-6aef193a90d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:15.306397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1218/2427] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 10:07:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:07:15.320975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93940c-9176-477b-b22c-bc09fae40d34 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:07:15.322977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ef93940c-9176-477b-b22c-bc09fae40d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:15.343075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93940c-9176-477b-b22c-bc09fae40d34 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:07:15.356427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540cf47-f7f9-4a02-95cf-c7ebcb084367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:07:15.357355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540cf47-f7f9-4a02-95cf-c7ebcb084367 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:15.357728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540cf47-f7f9-4a02-95cf-c7ebcb084367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:15.358096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540cf47-f7f9-4a02-95cf-c7ebcb084367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:15.369697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540cf47-f7f9-4a02-95cf-c7ebcb084367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:07:15.372237 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540cf47-f7f9-4a02-95cf-c7ebcb084367 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:07:15.384807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1210/2428] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:07:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:07:15.471594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faa959-2287-4332-8561-cd4ad3c8dfcc None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:15.471811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faa959-2287-4332-8561-cd4ad3c8dfcc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:15.472077 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faa959-2287-4332-8561-cd4ad3c8dfcc None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:15.473442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1219/2429] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 10 10:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:07:15.478801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96ed0c-6c0d-4200-9de2-a9bfdc49b4a3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:15.479154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96ed0c-6c0d-4200-9de2-a9bfdc49b4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:15.479431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96ed0c-6c0d-4200-9de2-a9bfdc49b4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:15.498932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-5d96ed0c-6c0d-4200-9de2-a9bfdc49b4a3 None None] MFA Rules not processed for user `904f7f036565419b9fdecc5e47acb45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:07:15.500145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-5d96ed0c-6c0d-4200-9de2-a9bfdc49b4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:15.510173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1211/2430] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 10 10:07:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:07:15.520401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faa959-2287-4332-8561-cd4ad3c8dfcc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:07:15.532001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faa959-2287-4332-8561-cd4ad3c8dfcc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:07:15.535547 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692f70f-e342-41dd-8d58-633724e2986c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:07:15.536136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f70f-e342-41dd-8d58-633724e2986c service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:15.536410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f70f-e342-41dd-8d58-633724e2986c service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:15.536676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f70f-e342-41dd-8d58-633724e2986c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:15.547082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692f70f-e342-41dd-8d58-633724e2986c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:07:15.548523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692f70f-e342-41dd-8d58-633724e2986c service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:07:15.559264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1220/2431] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:07:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:07:22.800762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e8d00-e845-43f3-8d6b-5b9ec6033e84 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:22.801621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e8d00-e845-43f3-8d6b-5b9ec6033e84 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:22.801899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e8d00-e845-43f3-8d6b-5b9ec6033e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:22.823001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e94e8d00-e845-43f3-8d6b-5b9ec6033e84 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:07:22.824243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e94e8d00-e845-43f3-8d6b-5b9ec6033e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:22.880549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1212/2432] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:07:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:07:22.892836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692f70f-e342-41dd-8d58-633724e2986c service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:07:22.894229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c692f70f-e342-41dd-8d58-633724e2986c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:22.964205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692f70f-e342-41dd-8d58-633724e2986c service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:07:22.969324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb70bb8-9cfe-4e13-a124-b30e16744d40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:07:22.969947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb70bb8-9cfe-4e13-a124-b30e16744d40 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:22.970246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb70bb8-9cfe-4e13-a124-b30e16744d40 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:22.970571 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb70bb8-9cfe-4e13-a124-b30e16744d40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:22.979358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb70bb8-9cfe-4e13-a124-b30e16744d40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:07:22.981010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb70bb8-9cfe-4e13-a124-b30e16744d40 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:07:22.989425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1221/2433] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:07:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:07:32.873550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=79 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:07:32.986172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=87 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:07:47.013309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bd2c0-42bc-4325-b415-e45686328a8a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:47.013809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bd2c0-42bc-4325-b415-e45686328a8a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:47.013905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bd2c0-42bc-4325-b415-e45686328a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:47.032464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-973bd2c0-42bc-4325-b415-e45686328a8a None None] MFA Rules not processed for user `befbca3738f0481296b63c70d6d2a797`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:07:47.033631 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-973bd2c0-42bc-4325-b415-e45686328a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:47.086286 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1213/2434] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:07:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:07:47.199595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9fb6-a8a5-4fbb-976d-3d88a16dc45c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:47.199595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9fb6-a8a5-4fbb-976d-3d88a16dc45c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:47.199595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9fb6-a8a5-4fbb-976d-3d88a16dc45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:47.220381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-876a9fb6-a8a5-4fbb-976d-3d88a16dc45c None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:07:47.221765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-876a9fb6-a8a5-4fbb-976d-3d88a16dc45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:07:47.232286 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1222/2435] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 10 10:07:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:07:47.249019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bd2c0-42bc-4325-b415-e45686328a8a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:07:47.262755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bd2c0-42bc-4325-b415-e45686328a8a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:07:47.269365 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf83f18-41d9-4220-8a52-215656af762e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:07:47.270213 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf83f18-41d9-4220-8a52-215656af762e service swift] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:07:47.270560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf83f18-41d9-4220-8a52-215656af762e service swift] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:07:47.270903 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf83f18-41d9-4220-8a52-215656af762e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:07:47.280411 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf83f18-41d9-4220-8a52-215656af762e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:07:47.282239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf83f18-41d9-4220-8a52-215656af762e service swift] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:07:47.297820 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1214/2436] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:07:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:07:57.216715 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:07:57.288377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=31 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:02.577252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a9fb6-a8a5-4fbb-976d-3d88a16dc45c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:02.586895 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a9fb6-a8a5-4fbb-976d-3d88a16dc45c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:02.590410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:02.591030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:02.591308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:02.591575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:02.593284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:02.606768 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:02.608204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:02.618800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1223/2437] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:07.195432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf83f18-41d9-4220-8a52-215656af762e service swift] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:07.206232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf83f18-41d9-4220-8a52-215656af762e service swift] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:07.211084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22ccd7-ebb1-4384-bc15-7382d9b9511b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:07.211816 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22ccd7-ebb1-4384-bc15-7382d9b9511b service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:07.212377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22ccd7-ebb1-4384-bc15-7382d9b9511b service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:07.212715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22ccd7-ebb1-4384-bc15-7382d9b9511b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:07.214700 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8d22ccd7-ebb1-4384-bc15-7382d9b9511b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:07.229329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22ccd7-ebb1-4384-bc15-7382d9b9511b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:07.230817 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22ccd7-ebb1-4384-bc15-7382d9b9511b service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:07.241931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1215/2438] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:10.022743 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:10.024133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:10.080749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05bb090-9c40-425e-a7d1-8ecc4df491da service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:10.086264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43aebc-0381-4a59-a4d6-74ff07450303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:10.087008 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43aebc-0381-4a59-a4d6-74ff07450303 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:10.087293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43aebc-0381-4a59-a4d6-74ff07450303 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:10.087599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43aebc-0381-4a59-a4d6-74ff07450303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:10.089435 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8a43aebc-0381-4a59-a4d6-74ff07450303 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:10.104423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43aebc-0381-4a59-a4d6-74ff07450303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:10.106712 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43aebc-0381-4a59-a4d6-74ff07450303 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:10.117029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1224/2439] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:10.755792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e512d-6f1a-411f-bc0d-4dd06ea1fc15 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:10.756229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e512d-6f1a-411f-bc0d-4dd06ea1fc15 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:10.756342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e512d-6f1a-411f-bc0d-4dd06ea1fc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:10.776841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-552e512d-6f1a-411f-bc0d-4dd06ea1fc15 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:10.778066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-552e512d-6f1a-411f-bc0d-4dd06ea1fc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:10.785458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1216/2440] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 10 10:08:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:10.813325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43aebc-0381-4a59-a4d6-74ff07450303 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:10.825316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43aebc-0381-4a59-a4d6-74ff07450303 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:10.830165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c52a73-9e43-4ec7-a539-4b165f6b4d30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:10.830749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c52a73-9e43-4ec7-a539-4b165f6b4d30 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:10.831057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c52a73-9e43-4ec7-a539-4b165f6b4d30 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:10.831325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c52a73-9e43-4ec7-a539-4b165f6b4d30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:10.839159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c52a73-9e43-4ec7-a539-4b165f6b4d30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:10.840711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c52a73-9e43-4ec7-a539-4b165f6b4d30 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:10.848533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1225/2441] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:08:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:08:17.700645 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e512d-6f1a-411f-bc0d-4dd06ea1fc15 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:17.702217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-552e512d-6f1a-411f-bc0d-4dd06ea1fc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:17.766808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c52a73-9e43-4ec7-a539-4b165f6b4d30 service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:17.768850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b9c52a73-9e43-4ec7-a539-4b165f6b4d30 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:17.788900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e512d-6f1a-411f-bc0d-4dd06ea1fc15 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:17.793918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5f9f7-237d-4da5-8a57-861f10d411d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:17.794568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5f9f7-237d-4da5-8a57-861f10d411d2 service placement] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:17.794857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5f9f7-237d-4da5-8a57-861f10d411d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:17.795152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5f9f7-237d-4da5-8a57-861f10d411d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:17.796881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-83a5f9f7-237d-4da5-8a57-861f10d411d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:17.809287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5f9f7-237d-4da5-8a57-861f10d411d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:17.810852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5f9f7-237d-4da5-8a57-861f10d411d2 service placement] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:17.820790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1217/2442] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:17.835836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c52a73-9e43-4ec7-a539-4b165f6b4d30 service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:17.840412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:17.841036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:17.841313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:17.841583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] PATH_INFO: `/v3/users/e9b57c80a45a454680de3e69db92c6cc` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:17.842696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b57c80a45a454680de3e69db92c6cc)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:17.844635 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:17.893067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b57c80a45a454680de3e69db92c6cc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:17.905214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b57c80a45a454680de3e69db92c6cc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:17.917479 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b57c80a45a454680de3e69db92c6cc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:17.918184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b57c80a45a454680de3e69db92c6cc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:17.918789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b57c80a45a454680de3e69db92c6cc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:17.922374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:17.933156 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1226/2443] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:08:17 2022] DELETE /identity/v3/users/e9b57c80a45a454680de3e69db92c6cc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:08:18.104572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5f9f7-237d-4da5-8a57-861f10d411d2 service placement] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:18.115321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5f9f7-237d-4da5-8a57-861f10d411d2 service placement] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:18.172236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:18.172857 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:18.173217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:18.173672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] PATH_INFO: `/v3/projects/281a4bf45d474217a84a31cee8128a87` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:18.174896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] RBAC: Authorizing `identity:delete_project(project_id=281a4bf45d474217a84a31cee8128a87)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:18.176805 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:18.225334 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281a4bf45d474217a84a31cee8128a87', 'request_id': 'req-1965066c-79d5-4686-8bab-569a312f733f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.231764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281a4bf45d474217a84a31cee8128a87', 'request_id': 'req-1965066c-79d5-4686-8bab-569a312f733f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.240055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281a4bf45d474217a84a31cee8128a87', 'request_id': 'req-1965066c-79d5-4686-8bab-569a312f733f'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.247720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:18.252368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] The token cache is being invalidate because project 281a4bf45d474217a84a31cee8128a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:08:18.252603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.253348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.256220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1218/2444] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:08:18 2022] DELETE /identity/v3/projects/281a4bf45d474217a84a31cee8128a87 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:08:18.260874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:18.262322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:18.276219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2a5c8-4398-454a-b84c-488410357b84 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:18.332060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:18.332687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:18.332964 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:18.333234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] PATH_INFO: `/v3/users/a49c7a9c2d4b40a68d2e478519a05b4f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:18.334344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49c7a9c2d4b40a68d2e478519a05b4f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:18.336296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:18.381264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49c7a9c2d4b40a68d2e478519a05b4f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.387903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49c7a9c2d4b40a68d2e478519a05b4f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.400003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49c7a9c2d4b40a68d2e478519a05b4f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.400754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49c7a9c2d4b40a68d2e478519a05b4f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.401437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b450209c-cd13-4a60-a575-263b9752ed16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49c7a9c2d4b40a68d2e478519a05b4f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.414560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1227/2445] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:08:18 2022] DELETE /identity/v3/users/a49c7a9c2d4b40a68d2e478519a05b4f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:08:18.666868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:18.668352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:18.682452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965066c-79d5-4686-8bab-569a312f733f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:18.745931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:18.746622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:18.746809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:18.747117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] PATH_INFO: `/v3/projects/21d02ef92a3a4da7b5ed3e589dbe78fd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:18.748202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d02ef92a3a4da7b5ed3e589dbe78fd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:18.750081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:18.798119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d02ef92a3a4da7b5ed3e589dbe78fd', 'request_id': 'req-8713fb98-5c2a-4fa1-a8bd-1715fd562627'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.804432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d02ef92a3a4da7b5ed3e589dbe78fd', 'request_id': 'req-8713fb98-5c2a-4fa1-a8bd-1715fd562627'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.811682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d02ef92a3a4da7b5ed3e589dbe78fd', 'request_id': 'req-8713fb98-5c2a-4fa1-a8bd-1715fd562627'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.822740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] The token cache is being invalidate because project 21d02ef92a3a4da7b5ed3e589dbe78fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:08:18.823053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.823674 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-8713fb98-5c2a-4fa1-a8bd-1715fd562627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:18.825152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1219/2446] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:08:18 2022] DELETE /identity/v3/projects/21d02ef92a3a4da7b5ed3e589dbe78fd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:08:18.830157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b78c-9568-4d71-8260-f6221dbf9411 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:18.830623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b78c-9568-4d71-8260-f6221dbf9411 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:18.830726 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b78c-9568-4d71-8260-f6221dbf9411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:18.851578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-29a7b78c-9568-4d71-8260-f6221dbf9411 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:18.852754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-29a7b78c-9568-4d71-8260-f6221dbf9411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:18.909424 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1228/2447] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:18.918067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90853cec-b92e-43c1-a804-80f63e080579 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:18.918351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90853cec-b92e-43c1-a804-80f63e080579 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:18.918620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90853cec-b92e-43c1-a804-80f63e080579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:18.938148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-90853cec-b92e-43c1-a804-80f63e080579 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:18.939353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-90853cec-b92e-43c1-a804-80f63e080579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:18.947674 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1220/2448] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:18.951794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7b78c-9568-4d71-8260-f6221dbf9411 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:18.961883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7b78c-9568-4d71-8260-f6221dbf9411 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:18.966943 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfdb0b-199a-470c-b9e8-9a97fe501278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:18.967566 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfdb0b-199a-470c-b9e8-9a97fe501278 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:18.967901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfdb0b-199a-470c-b9e8-9a97fe501278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:18.968169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfdb0b-199a-470c-b9e8-9a97fe501278 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:18.968805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfdb0b-199a-470c-b9e8-9a97fe501278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:08:18.969103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfdb0b-199a-470c-b9e8-9a97fe501278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:18.970669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfdb0b-199a-470c-b9e8-9a97fe501278 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:18.981986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1229/2449] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:08:18.986448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90853cec-b92e-43c1-a804-80f63e080579 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:18.995806 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90853cec-b92e-43c1-a804-80f63e080579 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:19.000878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f29f1-99bf-459c-968d-97f668f8d1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:19.001549 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f29f1-99bf-459c-968d-97f668f8d1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:19.001904 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f29f1-99bf-459c-968d-97f668f8d1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:19.002240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f29f1-99bf-459c-968d-97f668f8d1d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:19.002934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f29f1-99bf-459c-968d-97f668f8d1d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:19.005446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f29f1-99bf-459c-968d-97f668f8d1d7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:19.033534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1221/2450] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:08:18 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:08:19.037723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfdb0b-199a-470c-b9e8-9a97fe501278 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:19.043846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:19.049456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfdb0b-199a-470c-b9e8-9a97fe501278 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:19.108002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a334838-ba96-4c39-acfb-2e9db2e3d5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:19.108614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a334838-ba96-4c39-acfb-2e9db2e3d5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:19.108889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a334838-ba96-4c39-acfb-2e9db2e3d5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:19.109170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a334838-ba96-4c39-acfb-2e9db2e3d5d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:19.109789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a334838-ba96-4c39-acfb-2e9db2e3d5d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:19.111807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a334838-ba96-4c39-acfb-2e9db2e3d5d6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:19.135782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1230/2451] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:08:19 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:08:19.140073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f29f1-99bf-459c-968d-97f668f8d1d7 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:19.150326 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f29f1-99bf-459c-968d-97f668f8d1d7 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:19.154894 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0ae1c-7978-4f5c-b15d-860a7fa3a9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:19.155517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0ae1c-7978-4f5c-b15d-860a7fa3a9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:19.155795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0ae1c-7978-4f5c-b15d-860a7fa3a9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:19.156064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0ae1c-7978-4f5c-b15d-860a7fa3a9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:19.156702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc0ae1c-7978-4f5c-b15d-860a7fa3a9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:19.158318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc0ae1c-7978-4f5c-b15d-860a7fa3a9ad admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:19.168464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1222/2452] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:08:19.172373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a334838-ba96-4c39-acfb-2e9db2e3d5d6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:19.182565 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a334838-ba96-4c39-acfb-2e9db2e3d5d6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:19.187164 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590cbf0-2602-4cca-8f04-b2b5c67f698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:19.187752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590cbf0-2602-4cca-8f04-b2b5c67f698b admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:19.188029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590cbf0-2602-4cca-8f04-b2b5c67f698b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:19.188298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590cbf0-2602-4cca-8f04-b2b5c67f698b admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:19.188927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590cbf0-2602-4cca-8f04-b2b5c67f698b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:19.190553 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590cbf0-2602-4cca-8f04-b2b5c67f698b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:19.201532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1231/2453] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:08:19.205413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0ae1c-7978-4f5c-b15d-860a7fa3a9ad admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:19.215055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0ae1c-7978-4f5c-b15d-860a7fa3a9ad admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:19.219525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450aadb-60e3-49e8-bdf1-527865a2ba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:19.220174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450aadb-60e3-49e8-bdf1-527865a2ba50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:19.220449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450aadb-60e3-49e8-bdf1-527865a2ba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:19.220714 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450aadb-60e3-49e8-bdf1-527865a2ba50 admin admin] PATH_INFO: `/v3/projects/cdaf00b54b894a9db2de21ff1ef475a2/users/26782775a52b48e9a9a84d64d28ceca0/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:19.235608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450aadb-60e3-49e8-bdf1-527865a2ba50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdaf00b54b894a9db2de21ff1ef475a2, user_id=26782775a52b48e9a9a84d64d28ceca0, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:19.238207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450aadb-60e3-49e8-bdf1-527865a2ba50 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:19.251491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1223/2454] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:08:19 2022] PUT /identity/v3/projects/cdaf00b54b894a9db2de21ff1ef475a2/users/26782775a52b48e9a9a84d64d28ceca0/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:08:19.257278 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf98e7-ea9e-4d71-a617-674eccfd13e9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:19.257627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf98e7-ea9e-4d71-a617-674eccfd13e9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:19.257971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf98e7-ea9e-4d71-a617-674eccfd13e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:19.276661 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-18bf98e7-ea9e-4d71-a617-674eccfd13e9 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:19.277847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-18bf98e7-ea9e-4d71-a617-674eccfd13e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:19.334151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1232/2455] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:19.343560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450aadb-60e3-49e8-bdf1-527865a2ba50 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:19.345253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-9450aadb-60e3-49e8-bdf1-527865a2ba50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:19.375397 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:19.408460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450aadb-60e3-49e8-bdf1-527865a2ba50 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:19.412819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8a795-0eab-4f5a-a54e-3b71f9c7759a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:19.413810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8a795-0eab-4f5a-a54e-3b71f9c7759a service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:19.413810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8a795-0eab-4f5a-a54e-3b71f9c7759a service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:19.413997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8a795-0eab-4f5a-a54e-3b71f9c7759a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:19.423284 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8a795-0eab-4f5a-a54e-3b71f9c7759a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:19.424635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8a795-0eab-4f5a-a54e-3b71f9c7759a service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:19.434888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1224/2456] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:23.777058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d25e9d-c50b-4ecd-8f13-77990da62ad9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:23.777665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d25e9d-c50b-4ecd-8f13-77990da62ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:23.777665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d25e9d-c50b-4ecd-8f13-77990da62ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:23.797433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-93d25e9d-c50b-4ecd-8f13-77990da62ad9 None None] MFA Rules not processed for user `2dd3653b931643318f1a1b0820620277`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:23.798647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-93d25e9d-c50b-4ecd-8f13-77990da62ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:23.899003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1233/2457] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:24.495420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8a795-0eab-4f5a-a54e-3b71f9c7759a service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:24.504508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8a795-0eab-4f5a-a54e-3b71f9c7759a service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:24.509024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ec768-d844-4bcd-bba8-5848c6b91de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:24.510985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ec768-d844-4bcd-bba8-5848c6b91de0 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:24.510985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ec768-d844-4bcd-bba8-5848c6b91de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:24.510985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ec768-d844-4bcd-bba8-5848c6b91de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:24.511996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4b1ec768-d844-4bcd-bba8-5848c6b91de0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:24.526919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ec768-d844-4bcd-bba8-5848c6b91de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:24.528393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ec768-d844-4bcd-bba8-5848c6b91de0 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:24.538250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1225/2458] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:24.641426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03630814-f630-46a2-bc64-08d27ec98011 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:24.641952 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03630814-f630-46a2-bc64-08d27ec98011 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:24.642382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03630814-f630-46a2-bc64-08d27ec98011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:24.681906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-03630814-f630-46a2-bc64-08d27ec98011 None None] MFA Rules not processed for user `26782775a52b48e9a9a84d64d28ceca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:24.683294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-03630814-f630-46a2-bc64-08d27ec98011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:24.697706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:24.765357 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1234/2459] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:08:24.766498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ec768-d844-4bcd-bba8-5848c6b91de0 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:24.779380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ec768-d844-4bcd-bba8-5848c6b91de0 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:24.785388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9c1c6-7117-419f-ace9-e53f8df3344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:24.786131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9c1c6-7117-419f-ace9-e53f8df3344e admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:24.786476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9c1c6-7117-419f-ace9-e53f8df3344e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:24.786813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9c1c6-7117-419f-ace9-e53f8df3344e admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:24.787620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9c1c6-7117-419f-ace9-e53f8df3344e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:24.789978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9c1c6-7117-419f-ace9-e53f8df3344e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:24.826767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1226/2460] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:08:24 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:08:24.830891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03630814-f630-46a2-bc64-08d27ec98011 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:24.842760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03630814-f630-46a2-bc64-08d27ec98011 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:24.925540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3057158-8f54-4766-8fee-15e1be3f71f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:24.926346 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3057158-8f54-4766-8fee-15e1be3f71f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:24.926864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3057158-8f54-4766-8fee-15e1be3f71f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:24.928340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3057158-8f54-4766-8fee-15e1be3f71f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:24.929633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3057158-8f54-4766-8fee-15e1be3f71f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:24.931795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3057158-8f54-4766-8fee-15e1be3f71f7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:24.933962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9c1c6-7117-419f-ace9-e53f8df3344e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:24.935709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d0f9c1c6-7117-419f-ace9-e53f8df3344e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:24.959189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1235/2461] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:08:24 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:08:24.963981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3057158-8f54-4766-8fee-15e1be3f71f7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:24.974986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3057158-8f54-4766-8fee-15e1be3f71f7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:24.986441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e423c-24c5-4127-8459-ecbe8f57201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:24.987122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e423c-24c5-4127-8459-ecbe8f57201a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:24.987853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e423c-24c5-4127-8459-ecbe8f57201a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:24.987853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e423c-24c5-4127-8459-ecbe8f57201a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:24.988299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e423c-24c5-4127-8459-ecbe8f57201a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:24.990119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e423c-24c5-4127-8459-ecbe8f57201a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:25.001282 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1236/2462] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:08:25.005918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e423c-24c5-4127-8459-ecbe8f57201a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:25.016191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e423c-24c5-4127-8459-ecbe8f57201a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:25.021976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d9054-ff39-45fe-8c15-bddc7cd64a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:25.022636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d9054-ff39-45fe-8c15-bddc7cd64a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:25.022916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d9054-ff39-45fe-8c15-bddc7cd64a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:25.023216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d9054-ff39-45fe-8c15-bddc7cd64a28 admin admin] PATH_INFO: `/v3/projects/0a9e7a325ddc43cd811aba51fe2cc777/users/9fa5c90d55214eafa2f00117cd566c92/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:25.039031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9c1c6-7117-419f-ace9-e53f8df3344e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:25.040457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d9054-ff39-45fe-8c15-bddc7cd64a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9e7a325ddc43cd811aba51fe2cc777, user_id=9fa5c90d55214eafa2f00117cd566c92, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:25.043564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d9054-ff39-45fe-8c15-bddc7cd64a28 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:25.045067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4852b1-5976-4b9e-aadc-134ae94ef21e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:25.045829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4852b1-5976-4b9e-aadc-134ae94ef21e service placement] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:25.046154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4852b1-5976-4b9e-aadc-134ae94ef21e service placement] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:25.046506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4852b1-5976-4b9e-aadc-134ae94ef21e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:25.049144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6f4852b1-5976-4b9e-aadc-134ae94ef21e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:25.067783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1237/2463] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:08:25 2022] PUT /identity/v3/projects/0a9e7a325ddc43cd811aba51fe2cc777/users/9fa5c90d55214eafa2f00117cd566c92/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:08:25.070199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d9054-ff39-45fe-8c15-bddc7cd64a28 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:25.077429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4852b1-5976-4b9e-aadc-134ae94ef21e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:25.079154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4852b1-5976-4b9e-aadc-134ae94ef21e service placement] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:25.089593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1227/2464] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:08:25.136615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d9054-ff39-45fe-8c15-bddc7cd64a28 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:25.141655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e1042-b01c-4fee-9018-014413977003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:25.142261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e1042-b01c-4fee-9018-014413977003 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:25.142536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e1042-b01c-4fee-9018-014413977003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:25.142805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e1042-b01c-4fee-9018-014413977003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:25.143499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e1042-b01c-4fee-9018-014413977003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:25.145161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e1042-b01c-4fee-9018-014413977003 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:25.159309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1238/2465] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:08:25.159878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4852b1-5976-4b9e-aadc-134ae94ef21e service placement] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:25.170448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4852b1-5976-4b9e-aadc-134ae94ef21e service placement] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:25.175307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b17b4-c1fb-4bbe-9af2-910eb89a9a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:25.175983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b17b4-c1fb-4bbe-9af2-910eb89a9a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:25.176223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b17b4-c1fb-4bbe-9af2-910eb89a9a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:25.177019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b17b4-c1fb-4bbe-9af2-910eb89a9a8f admin admin] PATH_INFO: `/v3/projects/0a9e7a325ddc43cd811aba51fe2cc777/users/9fa5c90d55214eafa2f00117cd566c92/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:25.178930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b17b4-c1fb-4bbe-9af2-910eb89a9a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9e7a325ddc43cd811aba51fe2cc777, user_id=9fa5c90d55214eafa2f00117cd566c92, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:25.182728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b17b4-c1fb-4bbe-9af2-910eb89a9a8f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:25.199345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1228/2466] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:08:25 2022] PUT /identity/v3/projects/0a9e7a325ddc43cd811aba51fe2cc777/users/9fa5c90d55214eafa2f00117cd566c92/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:08:26.957287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e1042-b01c-4fee-9018-014413977003 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:26.960599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d10e1042-b01c-4fee-9018-014413977003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:26.972002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:27.047446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e1042-b01c-4fee-9018-014413977003 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:27.133849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73124cf0-e82d-40db-8f93-3d9bc7dcdf7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:27.134842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73124cf0-e82d-40db-8f93-3d9bc7dcdf7b service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:27.135331 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73124cf0-e82d-40db-8f93-3d9bc7dcdf7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:27.135755 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73124cf0-e82d-40db-8f93-3d9bc7dcdf7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:27.137725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-73124cf0-e82d-40db-8f93-3d9bc7dcdf7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:27.150491 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73124cf0-e82d-40db-8f93-3d9bc7dcdf7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:27.152067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73124cf0-e82d-40db-8f93-3d9bc7dcdf7b service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:27.160873 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1239/2467] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:30.831037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b17b4-c1fb-4bbe-9af2-910eb89a9a8f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:30.835024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-731b17b4-c1fb-4bbe-9af2-910eb89a9a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:30.982662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b17b4-c1fb-4bbe-9af2-910eb89a9a8f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:30.988008 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:30.988597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:30.989688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:30.989973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:30.991923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:31.043038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:31.103280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:31.104901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:31.115973 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1229/2468] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:31.293329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5dbff9-8a51-4053-95e9-4c917b970eaa None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:31.293783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5dbff9-8a51-4053-95e9-4c917b970eaa None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:31.293951 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5dbff9-8a51-4053-95e9-4c917b970eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:31.330334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3d5dbff9-8a51-4053-95e9-4c917b970eaa None None] MFA Rules not processed for user `9fa5c90d55214eafa2f00117cd566c92`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:31.331558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3d5dbff9-8a51-4053-95e9-4c917b970eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:31.429309 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1240/2469] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:31.459799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:31.461908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:31.540874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e04e7-e3b3-4174-9761-4a07c65be733 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:31.546696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6f43f-2e1a-4fe4-9205-103f92e3ab2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:31.547407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6f43f-2e1a-4fe4-9205-103f92e3ab2b service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:31.547748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6f43f-2e1a-4fe4-9205-103f92e3ab2b service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:31.548024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6f43f-2e1a-4fe4-9205-103f92e3ab2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:31.605215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6f43f-2e1a-4fe4-9205-103f92e3ab2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:31.606794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6f43f-2e1a-4fe4-9205-103f92e3ab2b service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:31.616255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1230/2470] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:32.892994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e84c0f-5bbf-4a14-82d4-65dbf149805e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:32.894037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e84c0f-5bbf-4a14-82d4-65dbf149805e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:32.898790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e84c0f-5bbf-4a14-82d4-65dbf149805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:32.941004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-23e84c0f-5bbf-4a14-82d4-65dbf149805e None None] MFA Rules not processed for user `26782775a52b48e9a9a84d64d28ceca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:32.943401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-23e84c0f-5bbf-4a14-82d4-65dbf149805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:32.952208 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1241/2471] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:34.050016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6f43f-2e1a-4fe4-9205-103f92e3ab2b service nova] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:34.088719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6f43f-2e1a-4fe4-9205-103f92e3ab2b service nova] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:34.095936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5e967-9c11-4217-a4d6-d7852a1d33fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:34.096783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5e967-9c11-4217-a4d6-d7852a1d33fe service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:34.097647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5e967-9c11-4217-a4d6-d7852a1d33fe service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:34.098240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5e967-9c11-4217-a4d6-d7852a1d33fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:34.100728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4ae5e967-9c11-4217-a4d6-d7852a1d33fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:34.114876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5e967-9c11-4217-a4d6-d7852a1d33fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:34.131340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5e967-9c11-4217-a4d6-d7852a1d33fe service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:34.162249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1231/2472] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:40.351807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97993fd-bf3e-4bbe-8179-02e4fdf037dc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:40.352291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97993fd-bf3e-4bbe-8179-02e4fdf037dc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:40.352544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97993fd-bf3e-4bbe-8179-02e4fdf037dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:40.373582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f97993fd-bf3e-4bbe-8179-02e4fdf037dc None None] MFA Rules not processed for user `2ef546ba84fa46d581ed6c82ec302e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:40.374934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f97993fd-bf3e-4bbe-8179-02e4fdf037dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:40.433740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:40.450628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1242/2473] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:41.297126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5e967-9c11-4217-a4d6-d7852a1d33fe service nova] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:41.306528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5e967-9c11-4217-a4d6-d7852a1d33fe service nova] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:41.310370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc63ba-2163-4637-b98a-8935eb61bb9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:41.311069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc63ba-2163-4637-b98a-8935eb61bb9d service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:41.311434 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc63ba-2163-4637-b98a-8935eb61bb9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:41.311768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc63ba-2163-4637-b98a-8935eb61bb9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:41.313527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b0fc63ba-2163-4637-b98a-8935eb61bb9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:41.324840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc63ba-2163-4637-b98a-8935eb61bb9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:41.326420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc63ba-2163-4637-b98a-8935eb61bb9d service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:41.340238 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1232/2474] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:42.014837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97993fd-bf3e-4bbe-8179-02e4fdf037dc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:42.016374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f97993fd-bf3e-4bbe-8179-02e4fdf037dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:42.059608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97993fd-bf3e-4bbe-8179-02e4fdf037dc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:42.068714 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ba63a-5569-4edc-9c8c-4f55e2e12794 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:42.069462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ba63a-5569-4edc-9c8c-4f55e2e12794 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:42.069837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ba63a-5569-4edc-9c8c-4f55e2e12794 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:42.070455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ba63a-5569-4edc-9c8c-4f55e2e12794 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:42.072631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-de6ba63a-5569-4edc-9c8c-4f55e2e12794 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:42.095371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ba63a-5569-4edc-9c8c-4f55e2e12794 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:42.097104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ba63a-5569-4edc-9c8c-4f55e2e12794 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:42.126881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1243/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:51.333291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=84 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:52.117031 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=11 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:52.817143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc63ba-2163-4637-b98a-8935eb61bb9d service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:52.838794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc63ba-2163-4637-b98a-8935eb61bb9d service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:52.842695 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ba63a-5569-4edc-9c8c-4f55e2e12794 service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:52.844596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:52.845344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:52.845690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:52.846047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:52.848149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:52.863776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:52.866248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:52.873780 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ba63a-5569-4edc-9c8c-4f55e2e12794 service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:52.880686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c020ee-a42d-49d7-a8fd-c1fcb17d54bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:52.882304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c020ee-a42d-49d7-a8fd-c1fcb17d54bd service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:52.882774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c020ee-a42d-49d7-a8fd-c1fcb17d54bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:52.883225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c020ee-a42d-49d7-a8fd-c1fcb17d54bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:52.885674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d6c020ee-a42d-49d7-a8fd-c1fcb17d54bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:52.891313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1233/2476] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:52.968473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c020ee-a42d-49d7-a8fd-c1fcb17d54bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:52.969912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c020ee-a42d-49d7-a8fd-c1fcb17d54bd service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:52.979811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1244/2477] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:08:56.874874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:56.877670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:56.898871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da97a9-dfe0-4125-a710-feff881ef3f0 service nova] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:56.904819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:56.905961 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:56.906872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:56.907354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] PATH_INFO: `/v3/users/befbca3738f0481296b63c70d6d2a797` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:56.908563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=befbca3738f0481296b63c70d6d2a797)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:56.923111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:56.996628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'befbca3738f0481296b63c70d6d2a797'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.005663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'befbca3738f0481296b63c70d6d2a797'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.013197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'befbca3738f0481296b63c70d6d2a797'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.027720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'befbca3738f0481296b63c70d6d2a797'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.028378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3c4beb30-e434-452c-9e63-86cf987426f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'befbca3738f0481296b63c70d6d2a797'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.036123 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1234/2478] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:08:56 2022] DELETE /identity/v3/users/befbca3738f0481296b63c70d6d2a797 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:08:57.234030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c020ee-a42d-49d7-a8fd-c1fcb17d54bd service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:57.247406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c020ee-a42d-49d7-a8fd-c1fcb17d54bd service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:57.313311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:57.314270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.314713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.315445 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] PATH_INFO: `/v3/projects/fbbf8047a09b4fee9a12288ef323338c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.316979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbbf8047a09b4fee9a12288ef323338c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:57.318906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:57.370161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbbf8047a09b4fee9a12288ef323338c', 'request_id': 'req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.377916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbbf8047a09b4fee9a12288ef323338c', 'request_id': 'req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.384639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbbf8047a09b4fee9a12288ef323338c', 'request_id': 'req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.397489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] The token cache is being invalidate because project fbbf8047a09b4fee9a12288ef323338c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:08:57.397779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.398433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ad6ee7c5-5a0f-44ef-9ab0-0e76e9b5a0d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:08:57.399968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1245/2479] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:08:57 2022] DELETE /identity/v3/projects/fbbf8047a09b4fee9a12288ef323338c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:08:57.411482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea5c47-35d8-4be3-9f48-3cfeec427734 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.411766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea5c47-35d8-4be3-9f48-3cfeec427734 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.412039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea5c47-35d8-4be3-9f48-3cfeec427734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.431925 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-cdea5c47-35d8-4be3-9f48-3cfeec427734 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:57.433118 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-cdea5c47-35d8-4be3-9f48-3cfeec427734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:57.493868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1235/2480] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:57.501938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae4fc-0d36-4240-ad75-daa4c005dbc6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.502216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae4fc-0d36-4240-ad75-daa4c005dbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.502662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae4fc-0d36-4240-ad75-daa4c005dbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.523886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3d9ae4fc-0d36-4240-ad75-daa4c005dbc6 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:57.525109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3d9ae4fc-0d36-4240-ad75-daa4c005dbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:57.533189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1246/2481] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:57.538180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea5c47-35d8-4be3-9f48-3cfeec427734 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:57.548333 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea5c47-35d8-4be3-9f48-3cfeec427734 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:57.553085 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820e467-2099-4ba9-b7d5-d9e599a46610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:57.553685 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820e467-2099-4ba9-b7d5-d9e599a46610 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.553993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820e467-2099-4ba9-b7d5-d9e599a46610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.554265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820e467-2099-4ba9-b7d5-d9e599a46610 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.554910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820e467-2099-4ba9-b7d5-d9e599a46610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:08:57.555320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820e467-2099-4ba9-b7d5-d9e599a46610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:57.556931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820e467-2099-4ba9-b7d5-d9e599a46610 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:57.568726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1236/2482] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:08:57.573037 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ae4fc-0d36-4240-ad75-daa4c005dbc6 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:57.582749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ae4fc-0d36-4240-ad75-daa4c005dbc6 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:57.587469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3fcb9-0502-45c4-a2dc-1482cf72cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:57.588075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3fcb9-0502-45c4-a2dc-1482cf72cf95 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.588354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3fcb9-0502-45c4-a2dc-1482cf72cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.588625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3fcb9-0502-45c4-a2dc-1482cf72cf95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.589247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3fcb9-0502-45c4-a2dc-1482cf72cf95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:57.591157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3fcb9-0502-45c4-a2dc-1482cf72cf95 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:57.606408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:57.620149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1247/2483] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:08:57 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:08:57.624357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820e467-2099-4ba9-b7d5-d9e599a46610 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:57.635692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820e467-2099-4ba9-b7d5-d9e599a46610 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:57.683970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:08:57.699900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e131ac-6b49-4ce6-8510-cdfce4f29fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:57.700515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e131ac-6b49-4ce6-8510-cdfce4f29fac admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.700793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e131ac-6b49-4ce6-8510-cdfce4f29fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.701066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e131ac-6b49-4ce6-8510-cdfce4f29fac admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.701693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e131ac-6b49-4ce6-8510-cdfce4f29fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:57.703653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e131ac-6b49-4ce6-8510-cdfce4f29fac admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:57.725971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1237/2484] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:08:57 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:08:57.730106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3fcb9-0502-45c4-a2dc-1482cf72cf95 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:57.740456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3fcb9-0502-45c4-a2dc-1482cf72cf95 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:57.745431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8bb25-3b24-45ad-a69d-024175e55aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:57.746118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8bb25-3b24-45ad-a69d-024175e55aac admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.746462 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8bb25-3b24-45ad-a69d-024175e55aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.746801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8bb25-3b24-45ad-a69d-024175e55aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.747550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8bb25-3b24-45ad-a69d-024175e55aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:57.749224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8bb25-3b24-45ad-a69d-024175e55aac admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:57.762203 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1248/2485] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:08:57.766004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e131ac-6b49-4ce6-8510-cdfce4f29fac admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:57.778167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e131ac-6b49-4ce6-8510-cdfce4f29fac admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:57.783220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20f189-3990-4234-b60e-103b7a0c6175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:57.783914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20f189-3990-4234-b60e-103b7a0c6175 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.784267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20f189-3990-4234-b60e-103b7a0c6175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.784633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20f189-3990-4234-b60e-103b7a0c6175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.785398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20f189-3990-4234-b60e-103b7a0c6175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:57.787114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20f189-3990-4234-b60e-103b7a0c6175 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:57.799994 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1238/2486] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:08:57.803933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8bb25-3b24-45ad-a69d-024175e55aac admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:57.814534 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8bb25-3b24-45ad-a69d-024175e55aac admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:57.820426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb49d-906f-4b21-937b-bc7ca1a271b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:57.821129 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb49d-906f-4b21-937b-bc7ca1a271b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.822178 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb49d-906f-4b21-937b-bc7ca1a271b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.822465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb49d-906f-4b21-937b-bc7ca1a271b6 admin admin] PATH_INFO: `/v3/projects/d006aeb4f532425ab423537ebcb2f0b1/users/67a34ddb337649ee948d319b0bf5fab3/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.838863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cb49d-906f-4b21-937b-bc7ca1a271b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d006aeb4f532425ab423537ebcb2f0b1, user_id=67a34ddb337649ee948d319b0bf5fab3, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:57.842004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cb49d-906f-4b21-937b-bc7ca1a271b6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:57.857368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1249/2487] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:08:57 2022] PUT /identity/v3/projects/d006aeb4f532425ab423537ebcb2f0b1/users/67a34ddb337649ee948d319b0bf5fab3/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:08:57.863589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53acd3-a454-49c6-a6f0-599a82997e63 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:57.863589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53acd3-a454-49c6-a6f0-599a82997e63 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:57.863871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53acd3-a454-49c6-a6f0-599a82997e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:57.883401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6c53acd3-a454-49c6-a6f0-599a82997e63 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:08:57.884601 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6c53acd3-a454-49c6-a6f0-599a82997e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:57.948264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1239/2488] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:08:57.958686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb49d-906f-4b21-937b-bc7ca1a271b6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:08:57.961128 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b52cb49d-906f-4b21-937b-bc7ca1a271b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:08:58.020434 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb49d-906f-4b21-937b-bc7ca1a271b6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:08:58.024369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d77c2-0559-46b3-8586-9c5d2875826f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:08:58.024982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d77c2-0559-46b3-8586-9c5d2875826f service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:08:58.025261 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d77c2-0559-46b3-8586-9c5d2875826f service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:08:58.025535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d77c2-0559-46b3-8586-9c5d2875826f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:08:58.035465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d77c2-0559-46b3-8586-9c5d2875826f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:08:58.036896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d77c2-0559-46b3-8586-9c5d2875826f service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:08:58.046622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1250/2489] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:02.213405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c35dc-93be-407a-b61a-7877e11a10ab None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:02.213876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c35dc-93be-407a-b61a-7877e11a10ab None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:02.213970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c35dc-93be-407a-b61a-7877e11a10ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:02.243424 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-201c35dc-93be-407a-b61a-7877e11a10ab None None] MFA Rules not processed for user `67a34ddb337649ee948d319b0bf5fab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:02.245520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-201c35dc-93be-407a-b61a-7877e11a10ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:02.321336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1240/2490] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:02.335791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d77c2-0559-46b3-8586-9c5d2875826f service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:02.337645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2b6d77c2-0559-46b3-8586-9c5d2875826f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:02.419010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d77c2-0559-46b3-8586-9c5d2875826f service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:02.432444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02554690-af7a-42b9-b3c1-326cd46a598d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:02.433374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554690-af7a-42b9-b3c1-326cd46a598d service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:02.433374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554690-af7a-42b9-b3c1-326cd46a598d service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:02.433578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554690-af7a-42b9-b3c1-326cd46a598d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:02.443122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02554690-af7a-42b9-b3c1-326cd46a598d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:02.444707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02554690-af7a-42b9-b3c1-326cd46a598d service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:02.454614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1251/2491] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:03.979153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f5549-4f5d-4fa0-b192-48dedecd6706 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:03.979644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f5549-4f5d-4fa0-b192-48dedecd6706 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:03.979734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f5549-4f5d-4fa0-b192-48dedecd6706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:04.001091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-b63f5549-4f5d-4fa0-b192-48dedecd6706 None None] MFA Rules not processed for user `67a34ddb337649ee948d319b0bf5fab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:04.002261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b63f5549-4f5d-4fa0-b192-48dedecd6706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:04.009652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1241/2492] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:05.656258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02554690-af7a-42b9-b3c1-326cd46a598d service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:05.657803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-02554690-af7a-42b9-b3c1-326cd46a598d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:05.670145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02554690-af7a-42b9-b3c1-326cd46a598d service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:05.673863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e18c6-847e-4c2e-b325-7565ffa5676f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:05.674460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e18c6-847e-4c2e-b325-7565ffa5676f service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:05.674737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e18c6-847e-4c2e-b325-7565ffa5676f service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:05.675048 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e18c6-847e-4c2e-b325-7565ffa5676f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:05.676792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-048e18c6-847e-4c2e-b325-7565ffa5676f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:05.730801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:05.754391 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e18c6-847e-4c2e-b325-7565ffa5676f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:05.756220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e18c6-847e-4c2e-b325-7565ffa5676f service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:05.768758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1252/2493] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:08.843761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42809215-f11b-4364-9dff-f57f04c5a277 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:08.844311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42809215-f11b-4364-9dff-f57f04c5a277 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:08.844400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42809215-f11b-4364-9dff-f57f04c5a277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:08.864171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-42809215-f11b-4364-9dff-f57f04c5a277 None None] MFA Rules not processed for user `7a9c654fc9bc4ad4b530d3f4df5c24c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:08.865630 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-42809215-f11b-4364-9dff-f57f04c5a277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:08.918186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:08.935542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1242/2494] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:15.763935 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=8 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:18.928261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:37.015418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e18c6-847e-4c2e-b325-7565ffa5676f service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:37.023868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e18c6-847e-4c2e-b325-7565ffa5676f service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:37.027426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883293c-138f-43e1-87c3-ec9243ca9b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:37.028017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883293c-138f-43e1-87c3-ec9243ca9b1e service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:37.028291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883293c-138f-43e1-87c3-ec9243ca9b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:37.028558 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883293c-138f-43e1-87c3-ec9243ca9b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:37.030231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b883293c-138f-43e1-87c3-ec9243ca9b1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:37.041465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883293c-138f-43e1-87c3-ec9243ca9b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:37.042860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883293c-138f-43e1-87c3-ec9243ca9b1e service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:37.052639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1253/2495] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:40.177471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42809215-f11b-4364-9dff-f57f04c5a277 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:40.179080 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-42809215-f11b-4364-9dff-f57f04c5a277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:40.193836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42809215-f11b-4364-9dff-f57f04c5a277 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:40.198346 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:40.198995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:40.199278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:40.199554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] PATH_INFO: `/v3/users/2dd3653b931643318f1a1b0820620277` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:40.200662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd3653b931643318f1a1b0820620277)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:40.202609 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:40.249996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd3653b931643318f1a1b0820620277'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.258174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd3653b931643318f1a1b0820620277'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.265589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd3653b931643318f1a1b0820620277'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.277481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd3653b931643318f1a1b0820620277'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.278257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2856fca3-d3e6-4e2a-98da-0b8effc53107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd3653b931643318f1a1b0820620277'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.284536 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1243/2496] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:09:40 2022] DELETE /identity/v3/users/2dd3653b931643318f1a1b0820620277 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:40.467764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883293c-138f-43e1-87c3-ec9243ca9b1e service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:40.480356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883293c-138f-43e1-87c3-ec9243ca9b1e service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:40.538747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:40.539405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:40.539685 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:40.539977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] PATH_INFO: `/v3/projects/66439eb22e4a4938b0522bfd356ca3f9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:40.541179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66439eb22e4a4938b0522bfd356ca3f9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:40.543127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:40.587936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66439eb22e4a4938b0522bfd356ca3f9', 'request_id': 'req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.594874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66439eb22e4a4938b0522bfd356ca3f9', 'request_id': 'req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.601648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66439eb22e4a4938b0522bfd356ca3f9', 'request_id': 'req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.611597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] The token cache is being invalidate because project 66439eb22e4a4938b0522bfd356ca3f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:09:40.611879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.612511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a9747c7d-58af-4c9e-9a13-7d8e06a1ece7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:40.613965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1254/2497] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:09:40 2022] DELETE /identity/v3/projects/66439eb22e4a4938b0522bfd356ca3f9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:40.619217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf77b6-f683-4a74-95c6-ce2314a1c0d2 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:40.619579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf77b6-f683-4a74-95c6-ce2314a1c0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:40.619946 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf77b6-f683-4a74-95c6-ce2314a1c0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:40.640789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-46bf77b6-f683-4a74-95c6-ce2314a1c0d2 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:40.641995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-46bf77b6-f683-4a74-95c6-ce2314a1c0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:40.701506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1244/2498] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:40.709897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea89a5-88c7-4ef4-a6fb-59304f0c9dc5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:40.710210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea89a5-88c7-4ef4-a6fb-59304f0c9dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:40.710482 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea89a5-88c7-4ef4-a6fb-59304f0c9dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:40.730668 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9eea89a5-88c7-4ef4-a6fb-59304f0c9dc5 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:40.731986 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9eea89a5-88c7-4ef4-a6fb-59304f0c9dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:40.740000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1255/2499] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:40.744362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf77b6-f683-4a74-95c6-ce2314a1c0d2 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:40.754279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf77b6-f683-4a74-95c6-ce2314a1c0d2 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:40.759190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e0429-bf9c-419f-ba14-6b09413299ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:40.759706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e0429-bf9c-419f-ba14-6b09413299ae admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:40.760351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e0429-bf9c-419f-ba14-6b09413299ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:40.760351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e0429-bf9c-419f-ba14-6b09413299ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:40.760855 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e0429-bf9c-419f-ba14-6b09413299ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:09:40.761154 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e0429-bf9c-419f-ba14-6b09413299ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:40.762838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e0429-bf9c-419f-ba14-6b09413299ae admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:40.774682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1245/2500] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:09:40.779016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea89a5-88c7-4ef4-a6fb-59304f0c9dc5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:40.789681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea89a5-88c7-4ef4-a6fb-59304f0c9dc5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:40.794430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87d9f7-b89a-41c8-b12a-d8574dfbd050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:40.795053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87d9f7-b89a-41c8-b12a-d8574dfbd050 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:40.795329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87d9f7-b89a-41c8-b12a-d8574dfbd050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:40.795599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87d9f7-b89a-41c8-b12a-d8574dfbd050 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:40.796298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87d9f7-b89a-41c8-b12a-d8574dfbd050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:40.798088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87d9f7-b89a-41c8-b12a-d8574dfbd050 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:40.825914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1256/2501] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:09:40 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:40.830261 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e0429-bf9c-419f-ba14-6b09413299ae admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:40.840837 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e0429-bf9c-419f-ba14-6b09413299ae admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:40.910088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bd98d-4b7e-41e5-ba44-f940a4ddfca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:40.910701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bd98d-4b7e-41e5-ba44-f940a4ddfca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:40.911005 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bd98d-4b7e-41e5-ba44-f940a4ddfca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:40.911287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bd98d-4b7e-41e5-ba44-f940a4ddfca2 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:40.911914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bd98d-4b7e-41e5-ba44-f940a4ddfca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:40.914382 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bd98d-4b7e-41e5-ba44-f940a4ddfca2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:40.923039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:40.940195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1246/2502] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:09:40 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:40.944224 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87d9f7-b89a-41c8-b12a-d8574dfbd050 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:40.955547 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87d9f7-b89a-41c8-b12a-d8574dfbd050 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:40.960132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3bd52-d602-4ddb-8080-fde5d6be858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:40.960794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3bd52-d602-4ddb-8080-fde5d6be858d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:40.961138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3bd52-d602-4ddb-8080-fde5d6be858d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:40.961693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3bd52-d602-4ddb-8080-fde5d6be858d admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:40.962427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3bd52-d602-4ddb-8080-fde5d6be858d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:40.964113 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3bd52-d602-4ddb-8080-fde5d6be858d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:40.976390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1257/2503] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:40.980461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bd98d-4b7e-41e5-ba44-f940a4ddfca2 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:40.990356 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bd98d-4b7e-41e5-ba44-f940a4ddfca2 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:40.996184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76f2aa-2261-450b-b2ab-f2898f26ac43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:40.996780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76f2aa-2261-450b-b2ab-f2898f26ac43 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:40.997064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76f2aa-2261-450b-b2ab-f2898f26ac43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:40.997338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76f2aa-2261-450b-b2ab-f2898f26ac43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:40.997978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76f2aa-2261-450b-b2ab-f2898f26ac43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:40.999621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76f2aa-2261-450b-b2ab-f2898f26ac43 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:41.014648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1247/2504] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:09:41.019317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3bd52-d602-4ddb-8080-fde5d6be858d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:41.029291 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3bd52-d602-4ddb-8080-fde5d6be858d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:41.033983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5b6d4-758d-4608-8429-ad3f3e1770dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:41.034625 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5b6d4-758d-4608-8429-ad3f3e1770dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:41.034910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5b6d4-758d-4608-8429-ad3f3e1770dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:41.035228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5b6d4-758d-4608-8429-ad3f3e1770dc admin admin] PATH_INFO: `/v3/projects/c2f7a7de5d7b4e3aa1236c6a2d7bb060/users/182fd8ff60f145fbbc743e4d07650499/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:41.049994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5b6d4-758d-4608-8429-ad3f3e1770dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f7a7de5d7b4e3aa1236c6a2d7bb060, user_id=182fd8ff60f145fbbc743e4d07650499, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:41.052654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5b6d4-758d-4608-8429-ad3f3e1770dc admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:41.065846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1258/2505] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:09:41 2022] PUT /identity/v3/projects/c2f7a7de5d7b4e3aa1236c6a2d7bb060/users/182fd8ff60f145fbbc743e4d07650499/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:09:41.076100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232141d-b6e9-42e8-881e-86d2a55da861 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:41.076386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232141d-b6e9-42e8-881e-86d2a55da861 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:41.076659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232141d-b6e9-42e8-881e-86d2a55da861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:41.103384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-1232141d-b6e9-42e8-881e-86d2a55da861 None None] MFA Rules not processed for user `182fd8ff60f145fbbc743e4d07650499`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:41.104563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1232141d-b6e9-42e8-881e-86d2a55da861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:41.167503 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1248/2506] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:41.179170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5b6d4-758d-4608-8429-ad3f3e1770dc admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:41.180999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5ed5b6d4-758d-4608-8429-ad3f3e1770dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:41.195354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:41.259222 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5b6d4-758d-4608-8429-ad3f3e1770dc admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:41.265019 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875b579-ef97-4f90-81a5-b49afacc5095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:41.265619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875b579-ef97-4f90-81a5-b49afacc5095 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:41.265897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875b579-ef97-4f90-81a5-b49afacc5095 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:41.266168 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875b579-ef97-4f90-81a5-b49afacc5095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:41.274287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875b579-ef97-4f90-81a5-b49afacc5095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:41.275887 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875b579-ef97-4f90-81a5-b49afacc5095 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:41.284946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1259/2507] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:48.572528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232141d-b6e9-42e8-881e-86d2a55da861 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:48.633309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232141d-b6e9-42e8-881e-86d2a55da861 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:48.637847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:48.638458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:48.638732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:48.639021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] PATH_INFO: `/v3/users/182fd8ff60f145fbbc743e4d07650499` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:48.640118 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=182fd8ff60f145fbbc743e4d07650499)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:48.642015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:48.687195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182fd8ff60f145fbbc743e4d07650499'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:48.694024 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182fd8ff60f145fbbc743e4d07650499'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:48.700840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182fd8ff60f145fbbc743e4d07650499'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:48.706448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:48.714446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182fd8ff60f145fbbc743e4d07650499'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:48.715281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182fd8ff60f145fbbc743e4d07650499'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:48.721639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1249/2508] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:09:48 2022] DELETE /identity/v3/users/182fd8ff60f145fbbc743e4d07650499 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:48.726172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49170fc8-b2e8-4b00-ab05-78be229ba5fc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:48.726622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49170fc8-b2e8-4b00-ab05-78be229ba5fc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:48.726737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49170fc8-b2e8-4b00-ab05-78be229ba5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:48.746574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-49170fc8-b2e8-4b00-ab05-78be229ba5fc None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:48.747734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-49170fc8-b2e8-4b00-ab05-78be229ba5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:48.811921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1260/2509] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:48.822400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:48.823835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:48.881620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7369a-3c75-4d56-ad3f-084a0853f834 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:48.885524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce634bb-a820-45a7-83a8-7088ffbf3a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:48.886126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce634bb-a820-45a7-83a8-7088ffbf3a39 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:48.886400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce634bb-a820-45a7-83a8-7088ffbf3a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:48.886668 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce634bb-a820-45a7-83a8-7088ffbf3a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:48.896997 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce634bb-a820-45a7-83a8-7088ffbf3a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:48.898423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce634bb-a820-45a7-83a8-7088ffbf3a39 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:48.907824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1250/2510] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:49.169137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49170fc8-b2e8-4b00-ab05-78be229ba5fc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:49.170853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-49170fc8-b2e8-4b00-ab05-78be229ba5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:49.183696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49170fc8-b2e8-4b00-ab05-78be229ba5fc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:49.188339 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:49.188963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.189238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.189508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] PATH_INFO: `/v3/projects/c2f7a7de5d7b4e3aa1236c6a2d7bb060` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.190663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f7a7de5d7b4e3aa1236c6a2d7bb060)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:49.192593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:49.238669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f7a7de5d7b4e3aa1236c6a2d7bb060', 'request_id': 'req-cceedf68-5ea6-4c07-9098-96f8799467f1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:49.245336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f7a7de5d7b4e3aa1236c6a2d7bb060', 'request_id': 'req-cceedf68-5ea6-4c07-9098-96f8799467f1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:49.251578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f7a7de5d7b4e3aa1236c6a2d7bb060', 'request_id': 'req-cceedf68-5ea6-4c07-9098-96f8799467f1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:49.261829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] The token cache is being invalidate because project c2f7a7de5d7b4e3aa1236c6a2d7bb060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:09:49.262046 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:49.262662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cceedf68-5ea6-4c07-9098-96f8799467f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:49.264115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1261/2511] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:09:49 2022] DELETE /identity/v3/projects/c2f7a7de5d7b4e3aa1236c6a2d7bb060 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:49.269041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89afc2c-4980-444b-b4b1-1d8d09cd2d83 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.269538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89afc2c-4980-444b-b4b1-1d8d09cd2d83 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.269905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89afc2c-4980-444b-b4b1-1d8d09cd2d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.289344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f89afc2c-4980-444b-b4b1-1d8d09cd2d83 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:49.290597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f89afc2c-4980-444b-b4b1-1d8d09cd2d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:49.347250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1251/2512] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:49.356014 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b23365-9612-4d7a-8d58-052cd1abecd7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.356299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b23365-9612-4d7a-8d58-052cd1abecd7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.356575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b23365-9612-4d7a-8d58-052cd1abecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.376488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b5b23365-9612-4d7a-8d58-052cd1abecd7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:49.377667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b5b23365-9612-4d7a-8d58-052cd1abecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:49.385232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1262/2513] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:49.389303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89afc2c-4980-444b-b4b1-1d8d09cd2d83 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:49.398865 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89afc2c-4980-444b-b4b1-1d8d09cd2d83 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:49.403417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34055c72-2c61-4b27-af59-d802d66b049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:49.404006 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34055c72-2c61-4b27-af59-d802d66b049e admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.404278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34055c72-2c61-4b27-af59-d802d66b049e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.404546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34055c72-2c61-4b27-af59-d802d66b049e admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.405260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34055c72-2c61-4b27-af59-d802d66b049e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:09:49.405557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34055c72-2c61-4b27-af59-d802d66b049e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:49.407160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34055c72-2c61-4b27-af59-d802d66b049e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:49.419062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1252/2514] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:09:49.423569 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b23365-9612-4d7a-8d58-052cd1abecd7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:49.433439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b23365-9612-4d7a-8d58-052cd1abecd7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:49.438041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f50311-5da5-4fa9-a023-9204759ef939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:49.438621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f50311-5da5-4fa9-a023-9204759ef939 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.438896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f50311-5da5-4fa9-a023-9204759ef939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.439190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f50311-5da5-4fa9-a023-9204759ef939 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.439806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f50311-5da5-4fa9-a023-9204759ef939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:49.441702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f50311-5da5-4fa9-a023-9204759ef939 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:49.456383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:49.468280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1263/2515] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:09:49 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:49.472998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34055c72-2c61-4b27-af59-d802d66b049e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:49.482793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34055c72-2c61-4b27-af59-d802d66b049e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:49.537822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3af7c-d78d-45a5-b8c9-e24db7c5599b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:49.538339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3af7c-d78d-45a5-b8c9-e24db7c5599b admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.538620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3af7c-d78d-45a5-b8c9-e24db7c5599b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.538890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3af7c-d78d-45a5-b8c9-e24db7c5599b admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.539538 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3af7c-d78d-45a5-b8c9-e24db7c5599b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:49.541437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3af7c-d78d-45a5-b8c9-e24db7c5599b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:49.561559 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1253/2516] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:09:49 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:49.565794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f50311-5da5-4fa9-a023-9204759ef939 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:49.576085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f50311-5da5-4fa9-a023-9204759ef939 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:49.581372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7dd03-74d4-455c-9a47-dd01cb329685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:49.581987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7dd03-74d4-455c-9a47-dd01cb329685 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.582259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7dd03-74d4-455c-9a47-dd01cb329685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.582523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7dd03-74d4-455c-9a47-dd01cb329685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.583194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7dd03-74d4-455c-9a47-dd01cb329685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:49.584828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7dd03-74d4-455c-9a47-dd01cb329685 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:49.594425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1264/2517] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:49.598803 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3af7c-d78d-45a5-b8c9-e24db7c5599b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:49.608441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3af7c-d78d-45a5-b8c9-e24db7c5599b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:49.612988 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0a7de-baad-4032-8644-bfed4c7250dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:49.613578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0a7de-baad-4032-8644-bfed4c7250dc admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.614344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0a7de-baad-4032-8644-bfed4c7250dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.614344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0a7de-baad-4032-8644-bfed4c7250dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.614747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0a7de-baad-4032-8644-bfed4c7250dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:49.616381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0a7de-baad-4032-8644-bfed4c7250dc admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:49.617605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:49.629173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1254/2518] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:49.633167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7dd03-74d4-455c-9a47-dd01cb329685 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:49.643630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7dd03-74d4-455c-9a47-dd01cb329685 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:49.648231 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3646e6d-7bc0-4227-afc3-29ae38eb0f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:49.648870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3646e6d-7bc0-4227-afc3-29ae38eb0f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.649146 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3646e6d-7bc0-4227-afc3-29ae38eb0f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.649415 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3646e6d-7bc0-4227-afc3-29ae38eb0f24 admin admin] PATH_INFO: `/v3/projects/bc1a0967d06e48d09b1d1ae452f71dbf/users/9a943e449a5e4be197970c3263d21d68/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.664737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3646e6d-7bc0-4227-afc3-29ae38eb0f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1a0967d06e48d09b1d1ae452f71dbf, user_id=9a943e449a5e4be197970c3263d21d68, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:49.667433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3646e6d-7bc0-4227-afc3-29ae38eb0f24 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:49.679518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1265/2519] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:09:49 2022] PUT /identity/v3/projects/bc1a0967d06e48d09b1d1ae452f71dbf/users/9a943e449a5e4be197970c3263d21d68/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:49.689679 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e20cce-7f97-49fe-ac62-0f1f67dc0a30 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.689966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e20cce-7f97-49fe-ac62-0f1f67dc0a30 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.690241 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e20cce-7f97-49fe-ac62-0f1f67dc0a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.717320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-13e20cce-7f97-49fe-ac62-0f1f67dc0a30 None None] MFA Rules not processed for user `9a943e449a5e4be197970c3263d21d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:49.718510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-13e20cce-7f97-49fe-ac62-0f1f67dc0a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:49.780889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1255/2520] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:49.792886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3646e6d-7bc0-4227-afc3-29ae38eb0f24 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:49.794563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d3646e6d-7bc0-4227-afc3-29ae38eb0f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:49.865748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3646e6d-7bc0-4227-afc3-29ae38eb0f24 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:49.871112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d829b5c-d6db-48ee-86f0-b946df9f6427 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:49.871704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d829b5c-d6db-48ee-86f0-b946df9f6427 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:49.871979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d829b5c-d6db-48ee-86f0-b946df9f6427 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:49.872251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d829b5c-d6db-48ee-86f0-b946df9f6427 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:49.880538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d829b5c-d6db-48ee-86f0-b946df9f6427 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:49.882098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d829b5c-d6db-48ee-86f0-b946df9f6427 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:49.890534 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1266/2521] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:51.153171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e20cce-7f97-49fe-ac62-0f1f67dc0a30 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:51.166497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e20cce-7f97-49fe-ac62-0f1f67dc0a30 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:51.172291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a2e36-28be-48f9-9c32-9464230ecbe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:51.172878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2e36-28be-48f9-9c32-9464230ecbe3 service nova] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:51.173159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2e36-28be-48f9-9c32-9464230ecbe3 service nova] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:51.173435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2e36-28be-48f9-9c32-9464230ecbe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:51.175193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-939a2e36-28be-48f9-9c32-9464230ecbe3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:51.234022 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a2e36-28be-48f9-9c32-9464230ecbe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:51.235660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a2e36-28be-48f9-9c32-9464230ecbe3 service nova] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:51.244947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1256/2522] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:52.571930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d829b5c-d6db-48ee-86f0-b946df9f6427 service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:52.632330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d829b5c-d6db-48ee-86f0-b946df9f6427 service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:52.636709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:52.637320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:52.637592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:52.637877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] PATH_INFO: `/v3/users/9a943e449a5e4be197970c3263d21d68` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:52.638945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a943e449a5e4be197970c3263d21d68)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:52.640870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:52.683536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:52.685087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a943e449a5e4be197970c3263d21d68'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:52.693816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a943e449a5e4be197970c3263d21d68'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:52.705767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a943e449a5e4be197970c3263d21d68'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:52.706414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a943e449a5e4be197970c3263d21d68'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:52.707033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a943e449a5e4be197970c3263d21d68'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:52.718990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1267/2523] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:09:52 2022] DELETE /identity/v3/users/9a943e449a5e4be197970c3263d21d68 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:52.724095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d74f1-9fc6-43e9-be32-e1c3a472a46b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:52.724546 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d74f1-9fc6-43e9-be32-e1c3a472a46b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:52.724666 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d74f1-9fc6-43e9-be32-e1c3a472a46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:52.743014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-688d74f1-9fc6-43e9-be32-e1c3a472a46b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:52.744180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-688d74f1-9fc6-43e9-be32-e1c3a472a46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:52.803770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1257/2524] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:52.813328 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:52.815300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:52.871666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9325a-05ca-4e6a-b416-915dde699234 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:52.875428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb09069-ce23-4f0e-8043-cf006f6556cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:52.876299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb09069-ce23-4f0e-8043-cf006f6556cc service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:52.876593 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb09069-ce23-4f0e-8043-cf006f6556cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:52.877436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb09069-ce23-4f0e-8043-cf006f6556cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:52.886348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb09069-ce23-4f0e-8043-cf006f6556cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:52.887794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb09069-ce23-4f0e-8043-cf006f6556cc service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:52.897678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1268/2525] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:53.167764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d74f1-9fc6-43e9-be32-e1c3a472a46b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:53.169347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-688d74f1-9fc6-43e9-be32-e1c3a472a46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:53.186782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d74f1-9fc6-43e9-be32-e1c3a472a46b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:53.193478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:53.194288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.194687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.195100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] PATH_INFO: `/v3/projects/bc1a0967d06e48d09b1d1ae452f71dbf` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.196388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1a0967d06e48d09b1d1ae452f71dbf)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:53.198539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:53.219212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:53.243498 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1a0967d06e48d09b1d1ae452f71dbf', 'request_id': 'req-2300dc3f-c007-4cf7-9994-d2c757a09445'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:53.249478 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1a0967d06e48d09b1d1ae452f71dbf', 'request_id': 'req-2300dc3f-c007-4cf7-9994-d2c757a09445'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:53.257135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1a0967d06e48d09b1d1ae452f71dbf', 'request_id': 'req-2300dc3f-c007-4cf7-9994-d2c757a09445'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:53.267322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] The token cache is being invalidate because project bc1a0967d06e48d09b1d1ae452f71dbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:09:53.267611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:53.268231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-2300dc3f-c007-4cf7-9994-d2c757a09445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:53.269713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1258/2526] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:09:53 2022] DELETE /identity/v3/projects/bc1a0967d06e48d09b1d1ae452f71dbf => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:53.274577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b35e21-a9bf-4584-8167-b044cedd9abe None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.274860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b35e21-a9bf-4584-8167-b044cedd9abe None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.275158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b35e21-a9bf-4584-8167-b044cedd9abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.293713 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-03b35e21-a9bf-4584-8167-b044cedd9abe None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:53.294889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-03b35e21-a9bf-4584-8167-b044cedd9abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:53.351308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1269/2527] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:53.358756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd62579-7f0e-4d84-bd39-dc57406720ad None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.359069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd62579-7f0e-4d84-bd39-dc57406720ad None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.359343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd62579-7f0e-4d84-bd39-dc57406720ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.379669 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-8dd62579-7f0e-4d84-bd39-dc57406720ad None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:53.380871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8dd62579-7f0e-4d84-bd39-dc57406720ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:53.388616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1259/2528] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:53.392568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b35e21-a9bf-4584-8167-b044cedd9abe None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:53.401972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b35e21-a9bf-4584-8167-b044cedd9abe None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:53.406458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692d365-5e77-49f8-b214-c4b201221cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:53.407081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d365-5e77-49f8-b214-c4b201221cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.407360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d365-5e77-49f8-b214-c4b201221cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.407629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d365-5e77-49f8-b214-c4b201221cc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.408269 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692d365-5e77-49f8-b214-c4b201221cc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:09:53.408564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692d365-5e77-49f8-b214-c4b201221cc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:53.410132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692d365-5e77-49f8-b214-c4b201221cc7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:53.421413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1270/2529] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:09:53.426205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd62579-7f0e-4d84-bd39-dc57406720ad None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:53.435851 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd62579-7f0e-4d84-bd39-dc57406720ad None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:53.441433 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c1c68-0032-4fe7-855e-79aaaf1bbf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:53.442017 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c1c68-0032-4fe7-855e-79aaaf1bbf5b admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.442289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c1c68-0032-4fe7-855e-79aaaf1bbf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.442560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c1c68-0032-4fe7-855e-79aaaf1bbf5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.443198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c1c68-0032-4fe7-855e-79aaaf1bbf5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:53.445137 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c1c68-0032-4fe7-855e-79aaaf1bbf5b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:53.472248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1260/2530] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:09:53 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:53.477265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692d365-5e77-49f8-b214-c4b201221cc7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:53.487698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692d365-5e77-49f8-b214-c4b201221cc7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:53.541784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:53.552005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072168ca-b3af-47cc-874a-9ac8f1d320ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:53.552674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072168ca-b3af-47cc-874a-9ac8f1d320ba admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.552954 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072168ca-b3af-47cc-874a-9ac8f1d320ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.553226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072168ca-b3af-47cc-874a-9ac8f1d320ba admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.553851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072168ca-b3af-47cc-874a-9ac8f1d320ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:53.555793 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072168ca-b3af-47cc-874a-9ac8f1d320ba admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:53.576294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1271/2531] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:09:53 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:53.581138 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c1c68-0032-4fe7-855e-79aaaf1bbf5b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:53.591911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c1c68-0032-4fe7-855e-79aaaf1bbf5b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:53.596748 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8726c-fe17-4989-9369-dd9e0b596724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:53.597465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8726c-fe17-4989-9369-dd9e0b596724 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.597745 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8726c-fe17-4989-9369-dd9e0b596724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.598042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8726c-fe17-4989-9369-dd9e0b596724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.598703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8726c-fe17-4989-9369-dd9e0b596724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:53.600465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8726c-fe17-4989-9369-dd9e0b596724 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:53.610688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1261/2532] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:53.614752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072168ca-b3af-47cc-874a-9ac8f1d320ba admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:53.624520 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072168ca-b3af-47cc-874a-9ac8f1d320ba admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:53.630298 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe227f-6642-48c5-b0a3-4a42a16e3c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:53.630965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe227f-6642-48c5-b0a3-4a42a16e3c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.631264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe227f-6642-48c5-b0a3-4a42a16e3c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.631557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe227f-6642-48c5-b0a3-4a42a16e3c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.632202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe227f-6642-48c5-b0a3-4a42a16e3c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:53.633888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe227f-6642-48c5-b0a3-4a42a16e3c3f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:53.645697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1272/2533] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:53.649882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8726c-fe17-4989-9369-dd9e0b596724 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:53.662269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8726c-fe17-4989-9369-dd9e0b596724 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:53.667285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23051053-5d48-4072-9289-ac09fc9767fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:53.667983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23051053-5d48-4072-9289-ac09fc9767fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.668265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23051053-5d48-4072-9289-ac09fc9767fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.668829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23051053-5d48-4072-9289-ac09fc9767fc admin admin] PATH_INFO: `/v3/projects/02699b30de4248d0af2c5992ca01330c/users/f6896385c5cc4f8387bfad299bdb7b03/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.686813 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23051053-5d48-4072-9289-ac09fc9767fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=02699b30de4248d0af2c5992ca01330c, user_id=f6896385c5cc4f8387bfad299bdb7b03, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:53.690265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23051053-5d48-4072-9289-ac09fc9767fc admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:53.703572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1262/2534] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:09:53 2022] PUT /identity/v3/projects/02699b30de4248d0af2c5992ca01330c/users/f6896385c5cc4f8387bfad299bdb7b03/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:53.715907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb60cc5-90d8-47f7-869f-0c9421285b43 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.716199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb60cc5-90d8-47f7-869f-0c9421285b43 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.716473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb60cc5-90d8-47f7-869f-0c9421285b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.743837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fcb60cc5-90d8-47f7-869f-0c9421285b43 None None] MFA Rules not processed for user `f6896385c5cc4f8387bfad299bdb7b03`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:53.745081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fcb60cc5-90d8-47f7-869f-0c9421285b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:53.809836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1273/2535] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:53.814890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23051053-5d48-4072-9289-ac09fc9767fc admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:53.882257 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23051053-5d48-4072-9289-ac09fc9767fc admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:53.890691 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53bba0-9dab-489e-a307-32dcc04fab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:53.895171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53bba0-9dab-489e-a307-32dcc04fab37 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.895448 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53bba0-9dab-489e-a307-32dcc04fab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.895716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53bba0-9dab-489e-a307-32dcc04fab37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.896332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53bba0-9dab-489e-a307-32dcc04fab37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:53.898208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53bba0-9dab-489e-a307-32dcc04fab37 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:53.925201 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1263/2536] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:09:53 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:53.929430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb60cc5-90d8-47f7-869f-0c9421285b43 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:53.940591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb60cc5-90d8-47f7-869f-0c9421285b43 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:53.997575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2bb67-d0f5-459a-bbbf-17ba575afeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:53.998172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2bb67-d0f5-459a-bbbf-17ba575afeac admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:53.998453 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2bb67-d0f5-459a-bbbf-17ba575afeac admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:53.998725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2bb67-d0f5-459a-bbbf-17ba575afeac admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:53.999370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2bb67-d0f5-459a-bbbf-17ba575afeac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:54.001537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2bb67-d0f5-459a-bbbf-17ba575afeac admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:54.021983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1274/2537] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:09:53 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:54.025992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53bba0-9dab-489e-a307-32dcc04fab37 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:54.036680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53bba0-9dab-489e-a307-32dcc04fab37 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:54.041339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91028cc-8c1b-4996-aae3-a8f79d9fcc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:54.041927 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91028cc-8c1b-4996-aae3-a8f79d9fcc7a admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:54.042202 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91028cc-8c1b-4996-aae3-a8f79d9fcc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:54.042472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91028cc-8c1b-4996-aae3-a8f79d9fcc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:54.043141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91028cc-8c1b-4996-aae3-a8f79d9fcc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:54.044718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91028cc-8c1b-4996-aae3-a8f79d9fcc7a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:54.055053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1264/2538] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:54.059228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2bb67-d0f5-459a-bbbf-17ba575afeac admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:54.069345 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2bb67-d0f5-459a-bbbf-17ba575afeac admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:54.074152 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb7456-188f-46b1-b8e8-97fadf6e33a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:54.074816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb7456-188f-46b1-b8e8-97fadf6e33a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:54.075158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb7456-188f-46b1-b8e8-97fadf6e33a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:54.075438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb7456-188f-46b1-b8e8-97fadf6e33a3 admin admin] PATH_INFO: `/v3/projects/8fdb4c1885e1429296c2831ad30a86be/users/35fb6f3685b447c6a9971b10923c9200/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:54.091022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb7456-188f-46b1-b8e8-97fadf6e33a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fdb4c1885e1429296c2831ad30a86be, user_id=35fb6f3685b447c6a9971b10923c9200, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:54.093662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb7456-188f-46b1-b8e8-97fadf6e33a3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:54.106774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1275/2539] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:09:54 2022] PUT /identity/v3/projects/8fdb4c1885e1429296c2831ad30a86be/users/35fb6f3685b447c6a9971b10923c9200/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:54.110671 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91028cc-8c1b-4996-aae3-a8f79d9fcc7a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:54.120418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91028cc-8c1b-4996-aae3-a8f79d9fcc7a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:54.132655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:54.178084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99721c8-7a41-47dc-915a-fe44dcb3a8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:54.178766 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99721c8-7a41-47dc-915a-fe44dcb3a8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:54.179114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99721c8-7a41-47dc-915a-fe44dcb3a8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:54.179407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99721c8-7a41-47dc-915a-fe44dcb3a8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:54.180051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99721c8-7a41-47dc-915a-fe44dcb3a8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:54.181647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99721c8-7a41-47dc-915a-fe44dcb3a8e3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:54.191386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1265/2540] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:54.195608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb7456-188f-46b1-b8e8-97fadf6e33a3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:54.206662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb7456-188f-46b1-b8e8-97fadf6e33a3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:54.211502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d86a0d-d3da-4c69-af15-51519500e54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:54.212161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d86a0d-d3da-4c69-af15-51519500e54c admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:54.212436 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d86a0d-d3da-4c69-af15-51519500e54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:54.212707 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d86a0d-d3da-4c69-af15-51519500e54c admin admin] PATH_INFO: `/v3/projects/8fdb4c1885e1429296c2831ad30a86be/users/35fb6f3685b447c6a9971b10923c9200/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:54.214617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d86a0d-d3da-4c69-af15-51519500e54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fdb4c1885e1429296c2831ad30a86be, user_id=35fb6f3685b447c6a9971b10923c9200, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:54.217276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d86a0d-d3da-4c69-af15-51519500e54c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:54.228374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1276/2541] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:09:54 2022] PUT /identity/v3/projects/8fdb4c1885e1429296c2831ad30a86be/users/35fb6f3685b447c6a9971b10923c9200/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:09:54.238683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec72d9-b1b8-437f-b6ef-fe662bedd4e9 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:54.239000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec72d9-b1b8-437f-b6ef-fe662bedd4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:54.239279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec72d9-b1b8-437f-b6ef-fe662bedd4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:54.268555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-dcec72d9-b1b8-437f-b6ef-fe662bedd4e9 None None] MFA Rules not processed for user `35fb6f3685b447c6a9971b10923c9200`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:54.269756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-dcec72d9-b1b8-437f-b6ef-fe662bedd4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:54.336960 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1266/2542] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:54.348259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d86a0d-d3da-4c69-af15-51519500e54c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:54.349949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-57d86a0d-d3da-4c69-af15-51519500e54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:54.364368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:54.408737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d86a0d-d3da-4c69-af15-51519500e54c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:54.412588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f1a4b-8f1f-4c28-ae62-f2c22d384a6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:54.412912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f1a4b-8f1f-4c28-ae62-f2c22d384a6b service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:54.413185 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f1a4b-8f1f-4c28-ae62-f2c22d384a6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:54.413455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f1a4b-8f1f-4c28-ae62-f2c22d384a6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:54.422331 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f1a4b-8f1f-4c28-ae62-f2c22d384a6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:54.423800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f1a4b-8f1f-4c28-ae62-f2c22d384a6b service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:54.432956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1277/2543] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:54.506198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec72d9-b1b8-437f-b6ef-fe662bedd4e9 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:54.515755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec72d9-b1b8-437f-b6ef-fe662bedd4e9 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:54.519959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034ef28-4b8c-4127-a663-9c5bd2756357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:54.520542 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034ef28-4b8c-4127-a663-9c5bd2756357 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:54.520814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034ef28-4b8c-4127-a663-9c5bd2756357 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:54.521080 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034ef28-4b8c-4127-a663-9c5bd2756357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:54.578775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034ef28-4b8c-4127-a663-9c5bd2756357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:54.580927 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034ef28-4b8c-4127-a663-9c5bd2756357 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:54.589774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1267/2544] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:57.304822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f1a4b-8f1f-4c28-ae62-f2c22d384a6b service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:57.364287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f1a4b-8f1f-4c28-ae62-f2c22d384a6b service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:57.368698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:57.369334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:57.369614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:57.369882 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] PATH_INFO: `/v3/users/f6896385c5cc4f8387bfad299bdb7b03` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:57.371015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6896385c5cc4f8387bfad299bdb7b03)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:57.372911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:57.418143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6896385c5cc4f8387bfad299bdb7b03'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.424794 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6896385c5cc4f8387bfad299bdb7b03'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.436180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6896385c5cc4f8387bfad299bdb7b03'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.436834 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6896385c5cc4f8387bfad299bdb7b03'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.437437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6896385c5cc4f8387bfad299bdb7b03'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.450704 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1278/2545] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:09:57 2022] DELETE /identity/v3/users/f6896385c5cc4f8387bfad299bdb7b03 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:57.455755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ad24a-e45d-4e27-b269-3faa5e602db1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:57.456200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ad24a-e45d-4e27-b269-3faa5e602db1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:57.456314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ad24a-e45d-4e27-b269-3faa5e602db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:57.474769 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-940ad24a-e45d-4e27-b269-3faa5e602db1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:57.475990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-940ad24a-e45d-4e27-b269-3faa5e602db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:57.508461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:57.535773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1268/2546] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:57.545451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:57.546823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:57.586772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:57.605068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63b7f4-12cd-44b7-8d25-d8bd94021498 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:57.608672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139c7b3-6fe6-4ae2-8dcb-b2ca0a748671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:57.609268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c7b3-6fe6-4ae2-8dcb-b2ca0a748671 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:57.609541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c7b3-6fe6-4ae2-8dcb-b2ca0a748671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:57.609807 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c7b3-6fe6-4ae2-8dcb-b2ca0a748671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:57.618412 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139c7b3-6fe6-4ae2-8dcb-b2ca0a748671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:57.619858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139c7b3-6fe6-4ae2-8dcb-b2ca0a748671 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:57.629864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1279/2547] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:09:57.898059 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ad24a-e45d-4e27-b269-3faa5e602db1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:57.899513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-940ad24a-e45d-4e27-b269-3faa5e602db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:57.911462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ad24a-e45d-4e27-b269-3faa5e602db1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:57.916650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:57.917314 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:57.917592 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:57.917929 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] PATH_INFO: `/v3/projects/02699b30de4248d0af2c5992ca01330c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:57.919002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02699b30de4248d0af2c5992ca01330c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:57.920900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:57.968964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02699b30de4248d0af2c5992ca01330c', 'request_id': 'req-bcc427fa-d121-4382-ad0c-79256d7f0415'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.975102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02699b30de4248d0af2c5992ca01330c', 'request_id': 'req-bcc427fa-d121-4382-ad0c-79256d7f0415'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.982304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02699b30de4248d0af2c5992ca01330c', 'request_id': 'req-bcc427fa-d121-4382-ad0c-79256d7f0415'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.992785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] The token cache is being invalidate because project 02699b30de4248d0af2c5992ca01330c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:09:57.993072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.993692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:57.995193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1269/2548] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:09:57 2022] DELETE /identity/v3/projects/02699b30de4248d0af2c5992ca01330c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:57.999139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139c7b3-6fe6-4ae2-8dcb-b2ca0a748671 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:58.009717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139c7b3-6fe6-4ae2-8dcb-b2ca0a748671 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:58.065285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:58.065907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.066191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.066467 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] PATH_INFO: `/v3/users/35fb6f3685b447c6a9971b10923c9200` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.067611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35fb6f3685b447c6a9971b10923c9200)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:58.069578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:58.114906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35fb6f3685b447c6a9971b10923c9200'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.121556 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35fb6f3685b447c6a9971b10923c9200'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.133127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35fb6f3685b447c6a9971b10923c9200'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.133781 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35fb6f3685b447c6a9971b10923c9200'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.134402 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b465fc4-3deb-4910-9e96-50c2e15b1c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35fb6f3685b447c6a9971b10923c9200'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.146267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1280/2549] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:09:57 2022] DELETE /identity/v3/users/35fb6f3685b447c6a9971b10923c9200 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:58.424571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:58.426147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:58.440413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc427fa-d121-4382-ad0c-79256d7f0415 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:58.505773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:58.506427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.506583 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.506852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] PATH_INFO: `/v3/projects/8fdb4c1885e1429296c2831ad30a86be` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.507993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fdb4c1885e1429296c2831ad30a86be)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:58.509911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:58.558456 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fdb4c1885e1429296c2831ad30a86be', 'request_id': 'req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.564641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fdb4c1885e1429296c2831ad30a86be', 'request_id': 'req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.571990 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fdb4c1885e1429296c2831ad30a86be', 'request_id': 'req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.573897 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:58.584347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] The token cache is being invalidate because project 8fdb4c1885e1429296c2831ad30a86be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:09:58.584638 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.585259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-58a2832b-3cc2-4902-a6b5-b0778b61ce0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:09:58.586753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1270/2550] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:09:58 2022] DELETE /identity/v3/projects/8fdb4c1885e1429296c2831ad30a86be => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:58.591783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ad86-e887-45d0-90e9-81babc95b890 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.592285 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ad86-e887-45d0-90e9-81babc95b890 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.592514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ad86-e887-45d0-90e9-81babc95b890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.612170 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1fa4ad86-e887-45d0-90e9-81babc95b890 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:58.613342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1fa4ad86-e887-45d0-90e9-81babc95b890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:58.662195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:58.675078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1281/2551] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:58.679689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcedb-f111-479b-be45-9c83f84e4e42 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.679972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcedb-f111-479b-be45-9c83f84e4e42 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.680240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcedb-f111-479b-be45-9c83f84e4e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.699299 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6c9dcedb-f111-479b-be45-9c83f84e4e42 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:58.700473 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6c9dcedb-f111-479b-be45-9c83f84e4e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:58.708684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1271/2552] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:58.712808 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ad86-e887-45d0-90e9-81babc95b890 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:58.722810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ad86-e887-45d0-90e9-81babc95b890 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:58.727347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba4631-8fc2-4852-b0ab-d2fd1e05bb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:58.727936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba4631-8fc2-4852-b0ab-d2fd1e05bb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.728214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba4631-8fc2-4852-b0ab-d2fd1e05bb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.728484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba4631-8fc2-4852-b0ab-d2fd1e05bb07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.729122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba4631-8fc2-4852-b0ab-d2fd1e05bb07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:09:58.729418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba4631-8fc2-4852-b0ab-d2fd1e05bb07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:58.731012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba4631-8fc2-4852-b0ab-d2fd1e05bb07 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:58.742092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1282/2553] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:09:58.746449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dcedb-f111-479b-be45-9c83f84e4e42 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:58.756330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dcedb-f111-479b-be45-9c83f84e4e42 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:58.760894 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686de53-d85e-45b0-8cf8-68a0200f11ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:58.761480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686de53-d85e-45b0-8cf8-68a0200f11ae admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.761756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686de53-d85e-45b0-8cf8-68a0200f11ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.762027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686de53-d85e-45b0-8cf8-68a0200f11ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.762645 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686de53-d85e-45b0-8cf8-68a0200f11ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:58.764593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686de53-d85e-45b0-8cf8-68a0200f11ae admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:58.791639 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1272/2554] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:09:58 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:58.795783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba4631-8fc2-4852-b0ab-d2fd1e05bb07 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:58.805727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba4631-8fc2-4852-b0ab-d2fd1e05bb07 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:58.860513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51c163-950f-4293-9617-fb7b564c7914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:58.861105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51c163-950f-4293-9617-fb7b564c7914 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.861380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51c163-950f-4293-9617-fb7b564c7914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.861646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51c163-950f-4293-9617-fb7b564c7914 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.862247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51c163-950f-4293-9617-fb7b564c7914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:58.864160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51c163-950f-4293-9617-fb7b564c7914 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:58.884897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1283/2555] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:09:58 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:58.889025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686de53-d85e-45b0-8cf8-68a0200f11ae admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:58.899906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686de53-d85e-45b0-8cf8-68a0200f11ae admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:58.904462 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11e065-be0f-45a3-befd-d638667f0464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:58.905053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11e065-be0f-45a3-befd-d638667f0464 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.905328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11e065-be0f-45a3-befd-d638667f0464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.905598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11e065-be0f-45a3-befd-d638667f0464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.906232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11e065-be0f-45a3-befd-d638667f0464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:58.907843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11e065-be0f-45a3-befd-d638667f0464 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:58.917476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1273/2556] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:09:58.921316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51c163-950f-4293-9617-fb7b564c7914 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:58.931203 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51c163-950f-4293-9617-fb7b564c7914 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:58.935763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f470ef-18f1-4751-9e43-9f3fd5dc035b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:58.936347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f470ef-18f1-4751-9e43-9f3fd5dc035b admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.936657 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f470ef-18f1-4751-9e43-9f3fd5dc035b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.936928 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f470ef-18f1-4751-9e43-9f3fd5dc035b admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.937559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f470ef-18f1-4751-9e43-9f3fd5dc035b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:58.939676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f470ef-18f1-4751-9e43-9f3fd5dc035b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:58.949886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1284/2557] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:58.953860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11e065-be0f-45a3-befd-d638667f0464 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:58.964508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11e065-be0f-45a3-befd-d638667f0464 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:58.969070 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fe4a6-64dc-46d1-bcfc-6be16d10ea56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:58.969735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe4a6-64dc-46d1-bcfc-6be16d10ea56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:58.970012 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe4a6-64dc-46d1-bcfc-6be16d10ea56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:58.970282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe4a6-64dc-46d1-bcfc-6be16d10ea56 admin admin] PATH_INFO: `/v3/projects/05da7481fac341f5829fbc3b2e03e30f/users/bb3c0a1a114e4dc98f5229e4afc7ea0d/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:58.984413 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fe4a6-64dc-46d1-bcfc-6be16d10ea56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05da7481fac341f5829fbc3b2e03e30f, user_id=bb3c0a1a114e4dc98f5229e4afc7ea0d, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:58.987083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fe4a6-64dc-46d1-bcfc-6be16d10ea56 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:58.999272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1274/2558] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:09:58 2022] PUT /identity/v3/projects/05da7481fac341f5829fbc3b2e03e30f/users/bb3c0a1a114e4dc98f5229e4afc7ea0d/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:59.011372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3f91-1b9f-4f4f-a9a6-7f6119b28b8b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:59.011597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3f91-1b9f-4f4f-a9a6-7f6119b28b8b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:59.011872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3f91-1b9f-4f4f-a9a6-7f6119b28b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:59.039051 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-85aa3f91-1b9f-4f4f-a9a6-7f6119b28b8b None None] MFA Rules not processed for user `bb3c0a1a114e4dc98f5229e4afc7ea0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:59.040244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-85aa3f91-1b9f-4f4f-a9a6-7f6119b28b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:59.102420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1285/2559] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:59.107115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fe4a6-64dc-46d1-bcfc-6be16d10ea56 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:59.167550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fe4a6-64dc-46d1-bcfc-6be16d10ea56 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:59.172115 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994971b3-865d-4f60-8904-10e6c8a06c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:59.172703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994971b3-865d-4f60-8904-10e6c8a06c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:59.172980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994971b3-865d-4f60-8904-10e6c8a06c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:59.173254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994971b3-865d-4f60-8904-10e6c8a06c6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:59.174394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994971b3-865d-4f60-8904-10e6c8a06c6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:59.176306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994971b3-865d-4f60-8904-10e6c8a06c6b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:59.202233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1275/2560] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:09:59 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:59.206295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa3f91-1b9f-4f4f-a9a6-7f6119b28b8b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:59.216719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa3f91-1b9f-4f4f-a9a6-7f6119b28b8b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:59.264665 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:59.274062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989087a-1f90-4e05-8ff5-587db120a552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:59.274691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989087a-1f90-4e05-8ff5-587db120a552 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:59.274992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989087a-1f90-4e05-8ff5-587db120a552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:59.275268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989087a-1f90-4e05-8ff5-587db120a552 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:59.275877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989087a-1f90-4e05-8ff5-587db120a552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:59.277763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989087a-1f90-4e05-8ff5-587db120a552 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:59.297974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1286/2561] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:09:59 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:09:59.302461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994971b3-865d-4f60-8904-10e6c8a06c6b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:59.312580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994971b3-865d-4f60-8904-10e6c8a06c6b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:59.317746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd8b04-4ada-4233-a5e9-07bf4047957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:59.318336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8b04-4ada-4233-a5e9-07bf4047957c admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:59.318614 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8b04-4ada-4233-a5e9-07bf4047957c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:59.318890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8b04-4ada-4233-a5e9-07bf4047957c admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:59.319567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd8b04-4ada-4233-a5e9-07bf4047957c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:59.321148 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd8b04-4ada-4233-a5e9-07bf4047957c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:59.331352 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1276/2562] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:59.335318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989087a-1f90-4e05-8ff5-587db120a552 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:59.344971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989087a-1f90-4e05-8ff5-587db120a552 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:59.349451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee263ce-1f23-4c74-b73c-c5b72c759da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:59.350084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee263ce-1f23-4c74-b73c-c5b72c759da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:59.350361 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee263ce-1f23-4c74-b73c-c5b72c759da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:59.350633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee263ce-1f23-4c74-b73c-c5b72c759da5 admin admin] PATH_INFO: `/v3/projects/c9cfed5a0c274b998010b766aa55f646/users/ccab4f8663e54a22b022091afdd28cc9/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:59.365300 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee263ce-1f23-4c74-b73c-c5b72c759da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9cfed5a0c274b998010b766aa55f646, user_id=ccab4f8663e54a22b022091afdd28cc9, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:59.368041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee263ce-1f23-4c74-b73c-c5b72c759da5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:59.381629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1287/2563] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:09:59 2022] PUT /identity/v3/projects/c9cfed5a0c274b998010b766aa55f646/users/ccab4f8663e54a22b022091afdd28cc9/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:59.385403 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd8b04-4ada-4233-a5e9-07bf4047957c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:59.395079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd8b04-4ada-4233-a5e9-07bf4047957c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:59.451612 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f8a11-a4c4-4cd6-af6f-f5e4ee8cb60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:59.452192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f8a11-a4c4-4cd6-af6f-f5e4ee8cb60b admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:59.452470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f8a11-a4c4-4cd6-af6f-f5e4ee8cb60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:59.452738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f8a11-a4c4-4cd6-af6f-f5e4ee8cb60b admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:59.453359 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f8a11-a4c4-4cd6-af6f-f5e4ee8cb60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:59.454930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f8a11-a4c4-4cd6-af6f-f5e4ee8cb60b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:59.456958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:09:59.465386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1277/2564] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:09:59.469776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee263ce-1f23-4c74-b73c-c5b72c759da5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:59.479963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee263ce-1f23-4c74-b73c-c5b72c759da5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:59.484560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844d040-60d7-493e-b080-3cefbdbfdc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:59.485213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844d040-60d7-493e-b080-3cefbdbfdc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:59.485486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844d040-60d7-493e-b080-3cefbdbfdc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:59.485752 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844d040-60d7-493e-b080-3cefbdbfdc1b admin admin] PATH_INFO: `/v3/projects/c9cfed5a0c274b998010b766aa55f646/users/ccab4f8663e54a22b022091afdd28cc9/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:59.488347 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844d040-60d7-493e-b080-3cefbdbfdc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9cfed5a0c274b998010b766aa55f646, user_id=ccab4f8663e54a22b022091afdd28cc9, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:59.491076 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844d040-60d7-493e-b080-3cefbdbfdc1b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:59.503929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1288/2565] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:09:59 2022] PUT /identity/v3/projects/c9cfed5a0c274b998010b766aa55f646/users/ccab4f8663e54a22b022091afdd28cc9/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:09:59.514157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651fbc7-4cc8-4037-a03c-755fd8f42393 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:59.514713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651fbc7-4cc8-4037-a03c-755fd8f42393 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:59.514939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651fbc7-4cc8-4037-a03c-755fd8f42393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:59.544196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-4651fbc7-4cc8-4037-a03c-755fd8f42393 None None] MFA Rules not processed for user `ccab4f8663e54a22b022091afdd28cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:09:59.545402 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-4651fbc7-4cc8-4037-a03c-755fd8f42393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:59.613918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1278/2566] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:09:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:09:59.624850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844d040-60d7-493e-b080-3cefbdbfdc1b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:09:59.626965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6844d040-60d7-493e-b080-3cefbdbfdc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:09:59.683728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844d040-60d7-493e-b080-3cefbdbfdc1b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:09:59.687316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e0c4d-d2cd-49f7-b08f-9e037ebbeb2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:09:59.687913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e0c4d-d2cd-49f7-b08f-9e037ebbeb2e service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:09:59.688191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e0c4d-d2cd-49f7-b08f-9e037ebbeb2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:09:59.688460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e0c4d-d2cd-49f7-b08f-9e037ebbeb2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:09:59.742968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e0c4d-d2cd-49f7-b08f-9e037ebbeb2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:09:59.744414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e0c4d-d2cd-49f7-b08f-9e037ebbeb2e service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:09:59.753296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1289/2567] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:09:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:10:00.888322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49719376-c8c2-43ab-9ec2-4a94a8aee496 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:00.888811 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49719376-c8c2-43ab-9ec2-4a94a8aee496 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:00.888901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49719376-c8c2-43ab-9ec2-4a94a8aee496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:00.908002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-49719376-c8c2-43ab-9ec2-4a94a8aee496 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:00.909157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-49719376-c8c2-43ab-9ec2-4a94a8aee496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:00.962372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1279/2568] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:10:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:01.008806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e0c4d-d2cd-49f7-b08f-9e037ebbeb2e service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:01.010223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-373e0c4d-d2cd-49f7-b08f-9e037ebbeb2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:01.080900 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:01.094217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e0c4d-d2cd-49f7-b08f-9e037ebbeb2e service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:01.100677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9621914-3a96-4c16-a718-4684e4fd49a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:01.101501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9621914-3a96-4c16-a718-4684e4fd49a0 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:01.101745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9621914-3a96-4c16-a718-4684e4fd49a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:01.102090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9621914-3a96-4c16-a718-4684e4fd49a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:01.112161 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9621914-3a96-4c16-a718-4684e4fd49a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:01.113985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9621914-3a96-4c16-a718-4684e4fd49a0 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:01.125007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1290/2569] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:10:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:10:05.444807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49719376-c8c2-43ab-9ec2-4a94a8aee496 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:05.446353 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-49719376-c8c2-43ab-9ec2-4a94a8aee496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:05.458181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49719376-c8c2-43ab-9ec2-4a94a8aee496 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:05.463204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee323e6-1e6f-43ea-89f2-2f15c1c14d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:05.463822 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee323e6-1e6f-43ea-89f2-2f15c1c14d14 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:05.464126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee323e6-1e6f-43ea-89f2-2f15c1c14d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:05.464400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee323e6-1e6f-43ea-89f2-2f15c1c14d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:05.474231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee323e6-1e6f-43ea-89f2-2f15c1c14d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:05.475697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee323e6-1e6f-43ea-89f2-2f15c1c14d14 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:05.487520 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1280/2570] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:10:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:10:11.119796 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:15.481579 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=66 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:21.258232 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b61759-86e5-4ba5-b123-de95b0001efd None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:21.258696 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b61759-86e5-4ba5-b123-de95b0001efd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:21.258947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b61759-86e5-4ba5-b123-de95b0001efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:21.280317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-08b61759-86e5-4ba5-b123-de95b0001efd None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:21.281579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-08b61759-86e5-4ba5-b123-de95b0001efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:21.289422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1291/2571] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:10:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:21.302391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee323e6-1e6f-43ea-89f2-2f15c1c14d14 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:21.319602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee323e6-1e6f-43ea-89f2-2f15c1c14d14 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:21.324856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58eeae-d610-4436-bf73-c8f38e6d6e8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:21.325453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58eeae-d610-4436-bf73-c8f38e6d6e8e service swift] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:21.325729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58eeae-d610-4436-bf73-c8f38e6d6e8e service swift] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:21.325998 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58eeae-d610-4436-bf73-c8f38e6d6e8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:21.335759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58eeae-d610-4436-bf73-c8f38e6d6e8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:21.337100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58eeae-d610-4436-bf73-c8f38e6d6e8e service swift] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:21.349628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1281/2572] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:10:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:10:30.657869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b61759-86e5-4ba5-b123-de95b0001efd None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:30.719889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b61759-86e5-4ba5-b123-de95b0001efd None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:30.724221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:30.724839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:30.725111 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:30.725377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] PATH_INFO: `/v3/users/bb3c0a1a114e4dc98f5229e4afc7ea0d` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:30.726486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3c0a1a114e4dc98f5229e4afc7ea0d)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:30.728425 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:30.775694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3c0a1a114e4dc98f5229e4afc7ea0d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:30.782413 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3c0a1a114e4dc98f5229e4afc7ea0d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:30.793902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3c0a1a114e4dc98f5229e4afc7ea0d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:30.794568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3c0a1a114e4dc98f5229e4afc7ea0d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:30.795204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3c0a1a114e4dc98f5229e4afc7ea0d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:30.808066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1292/2573] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:10:30 2022] DELETE /identity/v3/users/bb3c0a1a114e4dc98f5229e4afc7ea0d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:10:30.812477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c1ee8-093f-4a38-a559-2cc17022983e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:30.812947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c1ee8-093f-4a38-a559-2cc17022983e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:30.813053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c1ee8-093f-4a38-a559-2cc17022983e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:30.832129 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ce7c1ee8-093f-4a38-a559-2cc17022983e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:30.833373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ce7c1ee8-093f-4a38-a559-2cc17022983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:30.926682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1282/2574] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:30.940538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:30.942210 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:31.021525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d3dc7-0339-4359-8328-023df4c5a1de admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:31.025266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37f96d-5872-4506-9b4e-0773354e4e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:31.025885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37f96d-5872-4506-9b4e-0773354e4e66 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:31.026157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37f96d-5872-4506-9b4e-0773354e4e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:31.026428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37f96d-5872-4506-9b4e-0773354e4e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:31.036410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37f96d-5872-4506-9b4e-0773354e4e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:31.038861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37f96d-5872-4506-9b4e-0773354e4e66 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:31.055338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1293/2575] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:10:31.377212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c1ee8-093f-4a38-a559-2cc17022983e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:31.379955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ce7c1ee8-093f-4a38-a559-2cc17022983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:31.395387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c1ee8-093f-4a38-a559-2cc17022983e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:31.402621 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:31.403605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:31.404158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:31.404517 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] PATH_INFO: `/v3/projects/05da7481fac341f5829fbc3b2e03e30f` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:31.405933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05da7481fac341f5829fbc3b2e03e30f)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:31.408244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:31.458422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05da7481fac341f5829fbc3b2e03e30f', 'request_id': 'req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.464420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05da7481fac341f5829fbc3b2e03e30f', 'request_id': 'req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.471717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05da7481fac341f5829fbc3b2e03e30f', 'request_id': 'req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.486590 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] The token cache is being invalidate because project 05da7481fac341f5829fbc3b2e03e30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:10:31.487085 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.487871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.490749 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1283/2576] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:10:31 2022] DELETE /identity/v3/projects/05da7481fac341f5829fbc3b2e03e30f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:31.494839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37f96d-5872-4506-9b4e-0773354e4e66 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:31.505609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37f96d-5872-4506-9b4e-0773354e4e66 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:31.516666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:31.565783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:31.566540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:31.566828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:31.567142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] PATH_INFO: `/v3/users/ccab4f8663e54a22b022091afdd28cc9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:31.568272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccab4f8663e54a22b022091afdd28cc9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:31.570230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:31.617097 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccab4f8663e54a22b022091afdd28cc9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.624604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccab4f8663e54a22b022091afdd28cc9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.638212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccab4f8663e54a22b022091afdd28cc9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.638930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccab4f8663e54a22b022091afdd28cc9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.639708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d6216903-d110-4a74-8774-02f4ce973009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccab4f8663e54a22b022091afdd28cc9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:31.653619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1294/2577] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:10:31 2022] DELETE /identity/v3/users/ccab4f8663e54a22b022091afdd28cc9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:31.923163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:31.925073 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:31.943166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0cc09-68f6-4ad8-bd06-fa05fe3760a4 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.011093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.011598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.011871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.012139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] PATH_INFO: `/v3/projects/c9cfed5a0c274b998010b766aa55f646` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.013231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9cfed5a0c274b998010b766aa55f646)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.015178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.025962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:32.065143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9cfed5a0c274b998010b766aa55f646', 'request_id': 'req-4b0cb8eb-8445-46ca-9e6e-383349048fbc'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:32.070945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9cfed5a0c274b998010b766aa55f646', 'request_id': 'req-4b0cb8eb-8445-46ca-9e6e-383349048fbc'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:32.078459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9cfed5a0c274b998010b766aa55f646', 'request_id': 'req-4b0cb8eb-8445-46ca-9e6e-383349048fbc'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:32.088697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] The token cache is being invalidate because project c9cfed5a0c274b998010b766aa55f646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:10:32.088978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:32.089588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-4b0cb8eb-8445-46ca-9e6e-383349048fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:32.091055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1284/2578] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:10:31 2022] DELETE /identity/v3/projects/c9cfed5a0c274b998010b766aa55f646 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:32.096352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c935d-8c77-4841-b083-b758d16c7fbe None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.096815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c935d-8c77-4841-b083-b758d16c7fbe None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.096910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c935d-8c77-4841-b083-b758d16c7fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.118358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-876c935d-8c77-4841-b083-b758d16c7fbe None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:32.119607 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-876c935d-8c77-4841-b083-b758d16c7fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:32.176647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1295/2579] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:32.184390 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece8b05-0d25-4543-8441-28ba703b6774 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.184673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece8b05-0d25-4543-8441-28ba703b6774 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.184945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece8b05-0d25-4543-8441-28ba703b6774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.204982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-0ece8b05-0d25-4543-8441-28ba703b6774 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:32.206170 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0ece8b05-0d25-4543-8441-28ba703b6774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:32.213907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1285/2580] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:32.217930 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c935d-8c77-4841-b083-b758d16c7fbe None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.228141 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c935d-8c77-4841-b083-b758d16c7fbe None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.232739 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94c112-a33d-4892-87d5-b58f56291d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.233322 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94c112-a33d-4892-87d5-b58f56291d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.233599 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94c112-a33d-4892-87d5-b58f56291d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.233870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94c112-a33d-4892-87d5-b58f56291d62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.234514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94c112-a33d-4892-87d5-b58f56291d62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:10:32.234837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94c112-a33d-4892-87d5-b58f56291d62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.236451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94c112-a33d-4892-87d5-b58f56291d62 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.247612 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1296/2581] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:10:32.252067 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece8b05-0d25-4543-8441-28ba703b6774 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.260999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece8b05-0d25-4543-8441-28ba703b6774 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.267109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694f552-f980-4cf2-adf2-2d695866a358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.267633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694f552-f980-4cf2-adf2-2d695866a358 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.267905 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694f552-f980-4cf2-adf2-2d695866a358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.268172 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694f552-f980-4cf2-adf2-2d695866a358 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.268788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694f552-f980-4cf2-adf2-2d695866a358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.270655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694f552-f980-4cf2-adf2-2d695866a358 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.296934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1286/2582] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:10:32 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:10:32.301116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94c112-a33d-4892-87d5-b58f56291d62 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.310910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94c112-a33d-4892-87d5-b58f56291d62 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.329649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:32.367259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d374c-e2ac-4569-a567-6ec18fbfb790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.367858 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d374c-e2ac-4569-a567-6ec18fbfb790 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.368137 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d374c-e2ac-4569-a567-6ec18fbfb790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.368406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d374c-e2ac-4569-a567-6ec18fbfb790 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.369028 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d374c-e2ac-4569-a567-6ec18fbfb790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.370923 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d374c-e2ac-4569-a567-6ec18fbfb790 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.391784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1297/2583] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:10:32 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:10:32.396037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694f552-f980-4cf2-adf2-2d695866a358 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.406560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694f552-f980-4cf2-adf2-2d695866a358 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.411877 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6be428-d9cf-4364-b772-e908a4d8524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.412476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6be428-d9cf-4364-b772-e908a4d8524d admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.417211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6be428-d9cf-4364-b772-e908a4d8524d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.417437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6be428-d9cf-4364-b772-e908a4d8524d admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.418112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6be428-d9cf-4364-b772-e908a4d8524d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.419729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6be428-d9cf-4364-b772-e908a4d8524d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.430211 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1287/2584] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:10:32.434229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d374c-e2ac-4569-a567-6ec18fbfb790 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.443985 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d374c-e2ac-4569-a567-6ec18fbfb790 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.448532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f21e9-a215-4743-b3c4-902414d64a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.449118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f21e9-a215-4743-b3c4-902414d64a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.449428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f21e9-a215-4743-b3c4-902414d64a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.449698 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f21e9-a215-4743-b3c4-902414d64a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.450327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f21e9-a215-4743-b3c4-902414d64a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.451927 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f21e9-a215-4743-b3c4-902414d64a2f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.463294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1298/2585] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:10:32.467482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6be428-d9cf-4364-b772-e908a4d8524d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.477829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6be428-d9cf-4364-b772-e908a4d8524d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.482897 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b748b6-da8b-4654-bc9c-45d7729ae4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.483582 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b748b6-da8b-4654-bc9c-45d7729ae4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.483864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b748b6-da8b-4654-bc9c-45d7729ae4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.484145 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b748b6-da8b-4654-bc9c-45d7729ae4bd admin admin] PATH_INFO: `/v3/projects/f106d77c33834aae8bdc7224b65ac677/users/5606ad6f308845e4b601c646c486fe8f/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.499071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b748b6-da8b-4654-bc9c-45d7729ae4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f106d77c33834aae8bdc7224b65ac677, user_id=5606ad6f308845e4b601c646c486fe8f, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.502003 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b748b6-da8b-4654-bc9c-45d7729ae4bd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.514847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1288/2586] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:10:32 2022] PUT /identity/v3/projects/f106d77c33834aae8bdc7224b65ac677/users/5606ad6f308845e4b601c646c486fe8f/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:10:32.526059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06584f-d8c6-4e66-b152-335dfbbe5035 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.526409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06584f-d8c6-4e66-b152-335dfbbe5035 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.526760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06584f-d8c6-4e66-b152-335dfbbe5035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.555621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fa06584f-d8c6-4e66-b152-335dfbbe5035 None None] MFA Rules not processed for user `5606ad6f308845e4b601c646c486fe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:32.557043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fa06584f-d8c6-4e66-b152-335dfbbe5035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:32.620669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1299/2587] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:32.625112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b748b6-da8b-4654-bc9c-45d7729ae4bd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.685531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b748b6-da8b-4654-bc9c-45d7729ae4bd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.690643 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e76272-b6e8-4dc1-b28e-e4659c70b4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.691266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76272-b6e8-4dc1-b28e-e4659c70b4df admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.691543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76272-b6e8-4dc1-b28e-e4659c70b4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.691814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76272-b6e8-4dc1-b28e-e4659c70b4df admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.692418 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e76272-b6e8-4dc1-b28e-e4659c70b4df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.694318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e76272-b6e8-4dc1-b28e-e4659c70b4df admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.720058 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1289/2588] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:10:32 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:10:32.724055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06584f-d8c6-4e66-b152-335dfbbe5035 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.734622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06584f-d8c6-4e66-b152-335dfbbe5035 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.792461 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b720891-fa28-4b50-9711-c6efa79fb997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.793074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b720891-fa28-4b50-9711-c6efa79fb997 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.793355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b720891-fa28-4b50-9711-c6efa79fb997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.793654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b720891-fa28-4b50-9711-c6efa79fb997 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.794281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b720891-fa28-4b50-9711-c6efa79fb997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.796239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b720891-fa28-4b50-9711-c6efa79fb997 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.818645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1300/2589] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:10:32 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:10:32.822800 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e76272-b6e8-4dc1-b28e-e4659c70b4df admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.833724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e76272-b6e8-4dc1-b28e-e4659c70b4df admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.838719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4506b-7eb6-4c45-9395-727f5073409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.839366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4506b-7eb6-4c45-9395-727f5073409d admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.839644 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4506b-7eb6-4c45-9395-727f5073409d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.839915 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4506b-7eb6-4c45-9395-727f5073409d admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.840556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4506b-7eb6-4c45-9395-727f5073409d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.842178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4506b-7eb6-4c45-9395-727f5073409d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.853169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1290/2590] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:10:32.857458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b720891-fa28-4b50-9711-c6efa79fb997 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.867349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b720891-fa28-4b50-9711-c6efa79fb997 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.872165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc73dde-6c57-426d-b025-cc048ea8f198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.872828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc73dde-6c57-426d-b025-cc048ea8f198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.873112 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc73dde-6c57-426d-b025-cc048ea8f198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.873389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc73dde-6c57-426d-b025-cc048ea8f198 admin admin] PATH_INFO: `/v3/projects/f6a9a798b36e4197bdd90daeea56c478/users/4defb836ed0a4058bb14805106260a58/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.889006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc73dde-6c57-426d-b025-cc048ea8f198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a9a798b36e4197bdd90daeea56c478, user_id=4defb836ed0a4058bb14805106260a58, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.892417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc73dde-6c57-426d-b025-cc048ea8f198 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.895551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:32.908176 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1301/2591] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:10:32 2022] PUT /identity/v3/projects/f6a9a798b36e4197bdd90daeea56c478/users/4defb836ed0a4058bb14805106260a58/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:32.912091 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4506b-7eb6-4c45-9395-727f5073409d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:32.922932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4506b-7eb6-4c45-9395-727f5073409d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:32.930637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:32.981923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4e0b5-ecee-4daa-b1a0-c0bac57b5f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:32.982560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4e0b5-ecee-4daa-b1a0-c0bac57b5f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:32.982835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4e0b5-ecee-4daa-b1a0-c0bac57b5f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:32.983136 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4e0b5-ecee-4daa-b1a0-c0bac57b5f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:32.983764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4e0b5-ecee-4daa-b1a0-c0bac57b5f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:32.985376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4e0b5-ecee-4daa-b1a0-c0bac57b5f7b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:32.994792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1291/2592] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:10:32.998879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc73dde-6c57-426d-b025-cc048ea8f198 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:33.009108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc73dde-6c57-426d-b025-cc048ea8f198 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:33.013733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716f6a0-829d-4b72-b585-55f4fe0e9d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:33.014370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716f6a0-829d-4b72-b585-55f4fe0e9d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:33.014647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716f6a0-829d-4b72-b585-55f4fe0e9d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:33.014918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716f6a0-829d-4b72-b585-55f4fe0e9d26 admin admin] PATH_INFO: `/v3/projects/f6a9a798b36e4197bdd90daeea56c478/users/4defb836ed0a4058bb14805106260a58/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:33.017417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716f6a0-829d-4b72-b585-55f4fe0e9d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a9a798b36e4197bdd90daeea56c478, user_id=4defb836ed0a4058bb14805106260a58, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:33.020054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716f6a0-829d-4b72-b585-55f4fe0e9d26 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:33.030160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1302/2593] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:10:32 2022] PUT /identity/v3/projects/f6a9a798b36e4197bdd90daeea56c478/users/4defb836ed0a4058bb14805106260a58/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:33.041876 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414e8f-6ff1-41ab-bac3-6b616b6b27ba None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:33.042166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414e8f-6ff1-41ab-bac3-6b616b6b27ba None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:33.042441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414e8f-6ff1-41ab-bac3-6b616b6b27ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:33.068967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ed414e8f-6ff1-41ab-bac3-6b616b6b27ba None None] MFA Rules not processed for user `4defb836ed0a4058bb14805106260a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:33.070131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ed414e8f-6ff1-41ab-bac3-6b616b6b27ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:33.136633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1292/2594] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:33.148881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716f6a0-829d-4b72-b585-55f4fe0e9d26 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:33.151003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6716f6a0-829d-4b72-b585-55f4fe0e9d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:33.207816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716f6a0-829d-4b72-b585-55f4fe0e9d26 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:33.211485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cea93-1d31-4ec4-8173-ae9fed625ff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:33.212083 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cea93-1d31-4ec4-8173-ae9fed625ff4 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:33.212359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cea93-1d31-4ec4-8173-ae9fed625ff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:33.212630 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cea93-1d31-4ec4-8173-ae9fed625ff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:33.222001 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cea93-1d31-4ec4-8173-ae9fed625ff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:33.223473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cea93-1d31-4ec4-8173-ae9fed625ff4 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:33.232767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1303/2595] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:10:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:10:33.312887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414e8f-6ff1-41ab-bac3-6b616b6b27ba None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:33.322870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414e8f-6ff1-41ab-bac3-6b616b6b27ba None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:33.326378 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be90a04-ca08-46a4-89a8-32e0cd11d89d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:33.326982 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be90a04-ca08-46a4-89a8-32e0cd11d89d service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:33.327266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be90a04-ca08-46a4-89a8-32e0cd11d89d service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:33.327534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be90a04-ca08-46a4-89a8-32e0cd11d89d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:33.382773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be90a04-ca08-46a4-89a8-32e0cd11d89d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:33.384218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be90a04-ca08-46a4-89a8-32e0cd11d89d service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:33.393297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1293/2596] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:10:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:10:38.091881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cea93-1d31-4ec4-8173-ae9fed625ff4 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:38.151776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cea93-1d31-4ec4-8173-ae9fed625ff4 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:38.156227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:38.156838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:38.157113 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:38.157380 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] PATH_INFO: `/v3/users/5606ad6f308845e4b601c646c486fe8f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:38.158476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5606ad6f308845e4b601c646c486fe8f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:38.160394 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:38.204861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5606ad6f308845e4b601c646c486fe8f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.212003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5606ad6f308845e4b601c646c486fe8f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.224293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5606ad6f308845e4b601c646c486fe8f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.224949 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5606ad6f308845e4b601c646c486fe8f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.225549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5606ad6f308845e4b601c646c486fe8f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.237279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1304/2597] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:10:38 2022] DELETE /identity/v3/users/5606ad6f308845e4b601c646c486fe8f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:38.241673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c58c44-f5a7-41ec-a90b-527ad564f45a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:38.242165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c58c44-f5a7-41ec-a90b-527ad564f45a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:38.242260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c58c44-f5a7-41ec-a90b-527ad564f45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:38.260556 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c1c58c44-f5a7-41ec-a90b-527ad564f45a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:38.261715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c1c58c44-f5a7-41ec-a90b-527ad564f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:38.289150 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=100 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:38.321980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1294/2598] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:38.332615 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:38.334008 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:38.340504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:38.395919 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc564a7-affb-4079-b4ea-bfcbdd1c343c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:38.401364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720bff4-c718-4719-ab4a-1ef9494e8f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:38.401983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720bff4-c718-4719-ab4a-1ef9494e8f16 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:38.402265 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720bff4-c718-4719-ab4a-1ef9494e8f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:38.402538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720bff4-c718-4719-ab4a-1ef9494e8f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:38.413331 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720bff4-c718-4719-ab4a-1ef9494e8f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:38.414149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720bff4-c718-4719-ab4a-1ef9494e8f16 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:38.423899 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1305/2599] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:10:38.700122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c58c44-f5a7-41ec-a90b-527ad564f45a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:38.701597 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c1c58c44-f5a7-41ec-a90b-527ad564f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:38.714321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c58c44-f5a7-41ec-a90b-527ad564f45a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:38.719106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:38.719736 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:38.720010 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:38.720282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] PATH_INFO: `/v3/projects/f106d77c33834aae8bdc7224b65ac677` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:38.722053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f106d77c33834aae8bdc7224b65ac677)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:38.724118 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:38.782410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f106d77c33834aae8bdc7224b65ac677', 'request_id': 'req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.790095 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f106d77c33834aae8bdc7224b65ac677', 'request_id': 'req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.804715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f106d77c33834aae8bdc7224b65ac677', 'request_id': 'req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.804831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c464b-e1fa-4f44-b995-f4e15215931f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:38.805134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c464b-e1fa-4f44-b995-f4e15215931f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:38.805516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c464b-e1fa-4f44-b995-f4e15215931f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:38.816872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] The token cache is being invalidate because project f106d77c33834aae8bdc7224b65ac677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:10:38.817161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.817808 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.819417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1295/2600] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:10:38 2022] DELETE /identity/v3/projects/f106d77c33834aae8bdc7224b65ac677 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:38.823318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:38.824617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:38.826836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-629c464b-e1fa-4f44-b995-f4e15215931f None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:38.828106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-629c464b-e1fa-4f44-b995-f4e15215931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:38.883917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1306/2601] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 10 10:10:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:10:38.890847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8a3cf-00c4-4dca-b4d8-1748489799ad admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:38.896513 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:38.896744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c464b-e1fa-4f44-b995-f4e15215931f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:38.897150 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:38.897431 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:38.897707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] PATH_INFO: `/v3/users/4defb836ed0a4058bb14805106260a58` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:38.898080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-629c464b-e1fa-4f44-b995-f4e15215931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:38.898934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4defb836ed0a4058bb14805106260a58)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:38.900891 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:38.948522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4defb836ed0a4058bb14805106260a58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.956656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4defb836ed0a4058bb14805106260a58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.958673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:38.966663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4defb836ed0a4058bb14805106260a58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.969100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c464b-e1fa-4f44-b995-f4e15215931f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:38.975144 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54da5e-7703-4b9f-b334-cc1f2f89798c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:38.976103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54da5e-7703-4b9f-b334-cc1f2f89798c service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:38.976525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54da5e-7703-4b9f-b334-cc1f2f89798c service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:38.976800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54da5e-7703-4b9f-b334-cc1f2f89798c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:38.979569 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4defb836ed0a4058bb14805106260a58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.980363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4defb836ed0a4058bb14805106260a58'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:38.984828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54da5e-7703-4b9f-b334-cc1f2f89798c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:38.986445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1296/2602] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:10:38 2022] DELETE /identity/v3/users/4defb836ed0a4058bb14805106260a58 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:10:38.986573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54da5e-7703-4b9f-b334-cc1f2f89798c service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:38.994730 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1307/2603] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:10:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 10:10:39.275533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:39.277419 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:39.348835 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce90fe-9650-4119-a4e7-2bb9de75f60c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:39.353656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:39.354287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.354566 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.354836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] PATH_INFO: `/v3/projects/f6a9a798b36e4197bdd90daeea56c478` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.355978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a9a798b36e4197bdd90daeea56c478)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:39.357908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:39.412297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a9a798b36e4197bdd90daeea56c478', 'request_id': 'req-a3daea42-0eee-400c-af33-ea77d0b8a452'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:39.419198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a9a798b36e4197bdd90daeea56c478', 'request_id': 'req-a3daea42-0eee-400c-af33-ea77d0b8a452'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:39.427415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a9a798b36e4197bdd90daeea56c478', 'request_id': 'req-a3daea42-0eee-400c-af33-ea77d0b8a452'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:39.438001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] The token cache is being invalidate because project f6a9a798b36e4197bdd90daeea56c478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:10:39.438287 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:39.438903 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a3daea42-0eee-400c-af33-ea77d0b8a452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:39.440945 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1297/2604] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:10:39 2022] DELETE /identity/v3/projects/f6a9a798b36e4197bdd90daeea56c478 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:39.445735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32625-ef71-4048-8d64-a23d35d81b9a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.446209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32625-ef71-4048-8d64-a23d35d81b9a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.446302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32625-ef71-4048-8d64-a23d35d81b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.465917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-35b32625-ef71-4048-8d64-a23d35d81b9a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:39.467157 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-35b32625-ef71-4048-8d64-a23d35d81b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:39.516047 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:39.545836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1308/2605] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:39.553631 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c213a42-17c9-4631-8466-edb6892cf351 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.553921 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c213a42-17c9-4631-8466-edb6892cf351 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.554247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c213a42-17c9-4631-8466-edb6892cf351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.587153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-8c213a42-17c9-4631-8466-edb6892cf351 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:39.588436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8c213a42-17c9-4631-8466-edb6892cf351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:39.596440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1298/2606] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:39.600878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32625-ef71-4048-8d64-a23d35d81b9a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:39.611519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32625-ef71-4048-8d64-a23d35d81b9a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:39.616336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67b279-3877-4f68-885b-89d5abcfb28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:39.616936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67b279-3877-4f68-885b-89d5abcfb28c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.617212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67b279-3877-4f68-885b-89d5abcfb28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.617481 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67b279-3877-4f68-885b-89d5abcfb28c admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.618125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67b279-3877-4f68-885b-89d5abcfb28c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:10:39.618422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67b279-3877-4f68-885b-89d5abcfb28c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:39.620045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67b279-3877-4f68-885b-89d5abcfb28c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:39.632555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1309/2607] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:10:39.637174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c213a42-17c9-4631-8466-edb6892cf351 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:39.653922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c213a42-17c9-4631-8466-edb6892cf351 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:39.653922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586fda8-5826-4dbb-9731-e135fd0297cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:39.653922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586fda8-5826-4dbb-9731-e135fd0297cd admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.653922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586fda8-5826-4dbb-9731-e135fd0297cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.654336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586fda8-5826-4dbb-9731-e135fd0297cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.654336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586fda8-5826-4dbb-9731-e135fd0297cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:39.655307 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586fda8-5826-4dbb-9731-e135fd0297cd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:39.683184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1299/2608] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:10:39 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:10:39.687474 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67b279-3877-4f68-885b-89d5abcfb28c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:39.697942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67b279-3877-4f68-885b-89d5abcfb28c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:39.753197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2611dd-d6f0-404f-b9dd-46d9a50d0c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:39.753795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2611dd-d6f0-404f-b9dd-46d9a50d0c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.754074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2611dd-d6f0-404f-b9dd-46d9a50d0c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.754343 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2611dd-d6f0-404f-b9dd-46d9a50d0c83 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.755060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2611dd-d6f0-404f-b9dd-46d9a50d0c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:39.756953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2611dd-d6f0-404f-b9dd-46d9a50d0c83 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:39.777639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1310/2609] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:10:39 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:10:39.781729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586fda8-5826-4dbb-9731-e135fd0297cd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:39.791737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586fda8-5826-4dbb-9731-e135fd0297cd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:39.796319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c766d-b10d-46b8-b608-1c2ae42a7309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:39.796848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c766d-b10d-46b8-b608-1c2ae42a7309 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.797124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c766d-b10d-46b8-b608-1c2ae42a7309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.797394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c766d-b10d-46b8-b608-1c2ae42a7309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.798056 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c766d-b10d-46b8-b608-1c2ae42a7309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:39.800248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c766d-b10d-46b8-b608-1c2ae42a7309 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:39.810021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1300/2610] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:10:39.814051 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2611dd-d6f0-404f-b9dd-46d9a50d0c83 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:39.825440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2611dd-d6f0-404f-b9dd-46d9a50d0c83 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:39.830115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863a62d-153e-4ea0-8959-bcb44b81d6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:39.830843 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863a62d-153e-4ea0-8959-bcb44b81d6ef admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.831155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863a62d-153e-4ea0-8959-bcb44b81d6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.831428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863a62d-153e-4ea0-8959-bcb44b81d6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.832122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863a62d-153e-4ea0-8959-bcb44b81d6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:39.833683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863a62d-153e-4ea0-8959-bcb44b81d6ef admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:39.844526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1311/2611] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:10:39.848724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c766d-b10d-46b8-b608-1c2ae42a7309 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:39.858690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c766d-b10d-46b8-b608-1c2ae42a7309 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:39.863899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3913c09-a27c-440c-ad16-049a8d9c74f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:39.864551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3913c09-a27c-440c-ad16-049a8d9c74f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.864831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3913c09-a27c-440c-ad16-049a8d9c74f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.865111 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3913c09-a27c-440c-ad16-049a8d9c74f1 admin admin] PATH_INFO: `/v3/projects/a6c582a5eef54ab8aa5b835cb2d971e7/users/f164d3f12f5641bc9d016551969f015f/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.881114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3913c09-a27c-440c-ad16-049a8d9c74f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6c582a5eef54ab8aa5b835cb2d971e7, user_id=f164d3f12f5641bc9d016551969f015f, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:39.883844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3913c09-a27c-440c-ad16-049a8d9c74f1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:39.893400 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:39.900761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1301/2612] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:10:39 2022] PUT /identity/v3/projects/a6c582a5eef54ab8aa5b835cb2d971e7/users/f164d3f12f5641bc9d016551969f015f/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:39.911296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea023e-94a4-4a46-b6a3-9a620ac8c900 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:39.911584 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea023e-94a4-4a46-b6a3-9a620ac8c900 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:39.915237 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea023e-94a4-4a46-b6a3-9a620ac8c900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:39.944983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-efea023e-94a4-4a46-b6a3-9a620ac8c900 None None] MFA Rules not processed for user `f164d3f12f5641bc9d016551969f015f`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:39.946233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-efea023e-94a4-4a46-b6a3-9a620ac8c900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:40.009732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1312/2613] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:40.020245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3913c09-a27c-440c-ad16-049a8d9c74f1 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:40.021932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b3913c09-a27c-440c-ad16-049a8d9c74f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:40.076856 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3913c09-a27c-440c-ad16-049a8d9c74f1 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:40.080339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cf42d-1c31-4914-9d2d-00ecc54828d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:40.080928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cf42d-1c31-4914-9d2d-00ecc54828d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:40.081200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cf42d-1c31-4914-9d2d-00ecc54828d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:40.081467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cf42d-1c31-4914-9d2d-00ecc54828d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:40.090122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cf42d-1c31-4914-9d2d-00ecc54828d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:40.091577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cf42d-1c31-4914-9d2d-00ecc54828d9 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:40.100768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1302/2614] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:10:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:10:47.445916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea023e-94a4-4a46-b6a3-9a620ac8c900 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:47.485891 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:47.508377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea023e-94a4-4a46-b6a3-9a620ac8c900 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:47.512806 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:47.513420 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:47.513697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:47.513963 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] PATH_INFO: `/v3/users/f164d3f12f5641bc9d016551969f015f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:47.515099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f164d3f12f5641bc9d016551969f015f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:47.516990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:47.562088 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f164d3f12f5641bc9d016551969f015f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:47.569908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f164d3f12f5641bc9d016551969f015f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:47.582051 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f164d3f12f5641bc9d016551969f015f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:47.582803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f164d3f12f5641bc9d016551969f015f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:47.583505 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f164d3f12f5641bc9d016551969f015f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:47.596268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1313/2615] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:10:47 2022] DELETE /identity/v3/users/f164d3f12f5641bc9d016551969f015f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:47.600773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290e566-08ac-4773-8f38-6fcde49c3b2d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:47.601232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290e566-08ac-4773-8f38-6fcde49c3b2d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:47.601328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290e566-08ac-4773-8f38-6fcde49c3b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:47.619793 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-b290e566-08ac-4773-8f38-6fcde49c3b2d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:47.620969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b290e566-08ac-4773-8f38-6fcde49c3b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:47.678616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1303/2616] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:47.688237 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:47.690139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:47.745682 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2d450-cadd-491f-b844-2f23a62e1042 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:47.749367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a744a-114b-4551-8353-4f697426e35e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:47.750018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a744a-114b-4551-8353-4f697426e35e service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:47.750293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a744a-114b-4551-8353-4f697426e35e service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:47.751306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a744a-114b-4551-8353-4f697426e35e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:47.760785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a744a-114b-4551-8353-4f697426e35e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:47.762196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a744a-114b-4551-8353-4f697426e35e service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:47.772177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1314/2617] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:10:48.101519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290e566-08ac-4773-8f38-6fcde49c3b2d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:48.102885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b290e566-08ac-4773-8f38-6fcde49c3b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:48.114604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290e566-08ac-4773-8f38-6fcde49c3b2d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:48.120886 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:48.121515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.121797 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.122168 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] PATH_INFO: `/v3/projects/a6c582a5eef54ab8aa5b835cb2d971e7` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.123319 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6c582a5eef54ab8aa5b835cb2d971e7)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:48.125224 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:48.172717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6c582a5eef54ab8aa5b835cb2d971e7', 'request_id': 'req-b836a454-ac5e-464b-967f-a10754fe992c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:48.178768 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6c582a5eef54ab8aa5b835cb2d971e7', 'request_id': 'req-b836a454-ac5e-464b-967f-a10754fe992c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:48.185957 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6c582a5eef54ab8aa5b835cb2d971e7', 'request_id': 'req-b836a454-ac5e-464b-967f-a10754fe992c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:48.195907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] The token cache is being invalidate because project a6c582a5eef54ab8aa5b835cb2d971e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:10:48.196185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:48.196798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b836a454-ac5e-464b-967f-a10754fe992c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:10:48.198250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1304/2618] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:10:48 2022] DELETE /identity/v3/projects/a6c582a5eef54ab8aa5b835cb2d971e7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:48.203069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0104e-0114-46e1-8fca-9b8c7de5007b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.203362 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0104e-0114-46e1-8fca-9b8c7de5007b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.203641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0104e-0114-46e1-8fca-9b8c7de5007b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.223710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-47d0104e-0114-46e1-8fca-9b8c7de5007b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:48.224902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-47d0104e-0114-46e1-8fca-9b8c7de5007b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:48.274058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:48.283632 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1315/2619] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:48.290786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f2fe0-3460-44db-84a3-4de23291603d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.291097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f2fe0-3460-44db-84a3-4de23291603d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.291372 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f2fe0-3460-44db-84a3-4de23291603d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.310368 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-482f2fe0-3460-44db-84a3-4de23291603d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:48.311572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-482f2fe0-3460-44db-84a3-4de23291603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:48.319802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1305/2620] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:48.323790 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0104e-0114-46e1-8fca-9b8c7de5007b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:48.333753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0104e-0114-46e1-8fca-9b8c7de5007b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:48.338228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6992396-0d70-4e97-8a1b-24d6d19258ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:48.338821 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6992396-0d70-4e97-8a1b-24d6d19258ee admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.339126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6992396-0d70-4e97-8a1b-24d6d19258ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.339392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6992396-0d70-4e97-8a1b-24d6d19258ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.340022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6992396-0d70-4e97-8a1b-24d6d19258ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:10:48.340319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6992396-0d70-4e97-8a1b-24d6d19258ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:48.341888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6992396-0d70-4e97-8a1b-24d6d19258ee admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:48.353052 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1316/2621] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:10:48.357101 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f2fe0-3460-44db-84a3-4de23291603d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:48.366119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f2fe0-3460-44db-84a3-4de23291603d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:48.370608 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363f32f-1c03-41d5-9af1-7e27bb871ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:48.371215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363f32f-1c03-41d5-9af1-7e27bb871ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.371490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363f32f-1c03-41d5-9af1-7e27bb871ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.371753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363f32f-1c03-41d5-9af1-7e27bb871ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.372350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363f32f-1c03-41d5-9af1-7e27bb871ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:48.374216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363f32f-1c03-41d5-9af1-7e27bb871ef7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:48.395755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:48.404641 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1306/2622] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:10:48 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:10:48.408728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6992396-0d70-4e97-8a1b-24d6d19258ee admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:48.418642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6992396-0d70-4e97-8a1b-24d6d19258ee admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:48.474825 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c19e3-8186-4966-9d20-70c3da826246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:48.475457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c19e3-8186-4966-9d20-70c3da826246 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.475740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c19e3-8186-4966-9d20-70c3da826246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.476016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c19e3-8186-4966-9d20-70c3da826246 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.476629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c19e3-8186-4966-9d20-70c3da826246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:48.478531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c19e3-8186-4966-9d20-70c3da826246 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:48.499575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1317/2623] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:10:48 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:10:48.503911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363f32f-1c03-41d5-9af1-7e27bb871ef7 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:48.514135 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363f32f-1c03-41d5-9af1-7e27bb871ef7 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:48.518780 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476739-c30e-4a6d-b655-986527e0729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:48.519410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476739-c30e-4a6d-b655-986527e0729e admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.519690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476739-c30e-4a6d-b655-986527e0729e admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.519963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476739-c30e-4a6d-b655-986527e0729e admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.520602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476739-c30e-4a6d-b655-986527e0729e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:48.522178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476739-c30e-4a6d-b655-986527e0729e admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:48.531868 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1307/2624] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:10:48.536084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c19e3-8186-4966-9d20-70c3da826246 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:48.545709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c19e3-8186-4966-9d20-70c3da826246 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:48.550863 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c109201-30d9-42a5-b802-84d2ff2796bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:48.551468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c109201-30d9-42a5-b802-84d2ff2796bd admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.551747 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c109201-30d9-42a5-b802-84d2ff2796bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.552015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c109201-30d9-42a5-b802-84d2ff2796bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.552642 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c109201-30d9-42a5-b802-84d2ff2796bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:48.554199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c109201-30d9-42a5-b802-84d2ff2796bd admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:48.563995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1318/2625] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:10:48.568065 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476739-c30e-4a6d-b655-986527e0729e admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:48.577733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476739-c30e-4a6d-b655-986527e0729e admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:48.582315 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b12e7c-faac-43e3-930d-f600baf57052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:48.583036 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b12e7c-faac-43e3-930d-f600baf57052 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.583313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b12e7c-faac-43e3-930d-f600baf57052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.583580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b12e7c-faac-43e3-930d-f600baf57052 admin admin] PATH_INFO: `/v3/projects/e8ee5cfa7f0c40f2b5464878a82d337e/users/f5804adf2ac64022b3ac80991d1b26fd/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.598081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b12e7c-faac-43e3-930d-f600baf57052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8ee5cfa7f0c40f2b5464878a82d337e, user_id=f5804adf2ac64022b3ac80991d1b26fd, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:48.600722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b12e7c-faac-43e3-930d-f600baf57052 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:48.614559 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1308/2626] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:10:48 2022] PUT /identity/v3/projects/e8ee5cfa7f0c40f2b5464878a82d337e/users/f5804adf2ac64022b3ac80991d1b26fd/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:10:48.624706 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf977-ccef-4784-9e40-144d556cbd8e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.624938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf977-ccef-4784-9e40-144d556cbd8e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.625217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf977-ccef-4784-9e40-144d556cbd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.654697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d18bf977-ccef-4784-9e40-144d556cbd8e None None] MFA Rules not processed for user `f5804adf2ac64022b3ac80991d1b26fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:10:48.655942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d18bf977-ccef-4784-9e40-144d556cbd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:48.717532 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1319/2627] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:10:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:10:48.729465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b12e7c-faac-43e3-930d-f600baf57052 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:10:48.731151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b9b12e7c-faac-43e3-930d-f600baf57052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:10:48.786510 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b12e7c-faac-43e3-930d-f600baf57052 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:10:48.790131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f707af2-f83b-4d58-8e5f-bbb52fe14e96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8f456f0d15b4d068cfcd6476563247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:10:48.790717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f707af2-f83b-4d58-8e5f-bbb52fe14e96 service glance] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:10:48.791021 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f707af2-f83b-4d58-8e5f-bbb52fe14e96 service glance] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:10:48.791291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f707af2-f83b-4d58-8e5f-bbb52fe14e96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:10:48.801995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f707af2-f83b-4d58-8e5f-bbb52fe14e96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:10:48.803496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f707af2-f83b-4d58-8e5f-bbb52fe14e96 service glance] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:10:48.812907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1309/2628] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:10:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:10:58.714656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:10:58.808802 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:10.411260 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bf977-ccef-4784-9e40-144d556cbd8e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:10.412671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d18bf977-ccef-4784-9e40-144d556cbd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:10.495142 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bf977-ccef-4784-9e40-144d556cbd8e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:10.498579 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053b480-9f6c-48d4-8fac-fefcd3389e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:10.499195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053b480-9f6c-48d4-8fac-fefcd3389e30 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:10.499472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053b480-9f6c-48d4-8fac-fefcd3389e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:10.499741 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053b480-9f6c-48d4-8fac-fefcd3389e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:10.501670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4053b480-9f6c-48d4-8fac-fefcd3389e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:10.571526 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053b480-9f6c-48d4-8fac-fefcd3389e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:10.572941 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053b480-9f6c-48d4-8fac-fefcd3389e30 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:10.581654 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1320/2629] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:18.132947 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f707af2-f83b-4d58-8e5f-bbb52fe14e96 service glance] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:18.134348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3f707af2-f83b-4d58-8e5f-bbb52fe14e96 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:18.376322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f707af2-f83b-4d58-8e5f-bbb52fe14e96 service glance] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:18.382152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067103a-7184-478c-9d63-259697fed8e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:18.382687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067103a-7184-478c-9d63-259697fed8e3 service placement] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:18.382983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067103a-7184-478c-9d63-259697fed8e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:18.383266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067103a-7184-478c-9d63-259697fed8e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:18.384963 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7067103a-7184-478c-9d63-259697fed8e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:18.396193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067103a-7184-478c-9d63-259697fed8e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:18.397770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067103a-7184-478c-9d63-259697fed8e3 service placement] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:18.408032 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1310/2630] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:19.499721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086a764-a14f-4716-9e0b-c0526e0228fe None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:19.500221 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086a764-a14f-4716-9e0b-c0526e0228fe None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:19.500319 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086a764-a14f-4716-9e0b-c0526e0228fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:19.519032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4086a764-a14f-4716-9e0b-c0526e0228fe None None] MFA Rules not processed for user `7a9c654fc9bc4ad4b530d3f4df5c24c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:19.520225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4086a764-a14f-4716-9e0b-c0526e0228fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:19.526972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1321/2631] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:24.184476 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067103a-7184-478c-9d63-259697fed8e3 service placement] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:24.265753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067103a-7184-478c-9d63-259697fed8e3 service placement] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:24.271276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:24.272167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:24.272600 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:24.273023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] PATH_INFO: `/v3/users/f5804adf2ac64022b3ac80991d1b26fd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:24.274332 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5804adf2ac64022b3ac80991d1b26fd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:24.276578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:24.303650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:24.330035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5804adf2ac64022b3ac80991d1b26fd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.337313 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5804adf2ac64022b3ac80991d1b26fd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.344389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5804adf2ac64022b3ac80991d1b26fd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.356015 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5804adf2ac64022b3ac80991d1b26fd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.356710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5804adf2ac64022b3ac80991d1b26fd'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.362971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1311/2632] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:11:24 2022] DELETE /identity/v3/users/f5804adf2ac64022b3ac80991d1b26fd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:24.367578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42889-9036-4af8-bd32-b954915d0181 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:24.368089 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42889-9036-4af8-bd32-b954915d0181 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:24.368721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42889-9036-4af8-bd32-b954915d0181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:24.424110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-23e42889-9036-4af8-bd32-b954915d0181 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:24.425828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-23e42889-9036-4af8-bd32-b954915d0181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:24.474358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:24.490205 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1322/2633] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:11:24.500052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:24.501789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:24.578256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def9f5-6268-4198-8347-a80c687446b5 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:24.581893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089756c1-56c1-4a90-bf02-0d27d630d395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:24.582483 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089756c1-56c1-4a90-bf02-0d27d630d395 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:24.582755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089756c1-56c1-4a90-bf02-0d27d630d395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:24.583054 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089756c1-56c1-4a90-bf02-0d27d630d395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:24.591701 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089756c1-56c1-4a90-bf02-0d27d630d395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:24.593096 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089756c1-56c1-4a90-bf02-0d27d630d395 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:24.602053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1312/2634] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:11:24.882515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e42889-9036-4af8-bd32-b954915d0181 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:24.883976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-23e42889-9036-4af8-bd32-b954915d0181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:24.895903 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e42889-9036-4af8-bd32-b954915d0181 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:24.900956 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:24.901588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:24.901922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:24.902193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] PATH_INFO: `/v3/projects/e8ee5cfa7f0c40f2b5464878a82d337e` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:24.903330 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8ee5cfa7f0c40f2b5464878a82d337e)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:24.905249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:24.952443 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8ee5cfa7f0c40f2b5464878a82d337e', 'request_id': 'req-9a3d4556-16a6-40b2-908a-09b9877be63d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.959733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8ee5cfa7f0c40f2b5464878a82d337e', 'request_id': 'req-9a3d4556-16a6-40b2-908a-09b9877be63d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.966080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8ee5cfa7f0c40f2b5464878a82d337e', 'request_id': 'req-9a3d4556-16a6-40b2-908a-09b9877be63d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.976886 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] The token cache is being invalidate because project e8ee5cfa7f0c40f2b5464878a82d337e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:11:24.977288 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.977920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:24.979426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1323/2635] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:11:24 2022] DELETE /identity/v3/projects/e8ee5cfa7f0c40f2b5464878a82d337e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:34.598046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=55,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:34.973877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:36.093676 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089756c1-56c1-4a90-bf02-0d27d630d395 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:36.103057 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089756c1-56c1-4a90-bf02-0d27d630d395 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:36.149569 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc422a8-aa82-4db2-b284-ede373a73bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:36.150150 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc422a8-aa82-4db2-b284-ede373a73bae service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:36.150420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc422a8-aa82-4db2-b284-ede373a73bae service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:36.150684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc422a8-aa82-4db2-b284-ede373a73bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:36.152416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-edc422a8-aa82-4db2-b284-ede373a73bae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:36.212922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc422a8-aa82-4db2-b284-ede373a73bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:36.214363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc422a8-aa82-4db2-b284-ede373a73bae service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:36.224101 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1313/2636] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:39.597264 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:39.598878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:39.628415 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d4556-16a6-40b2-908a-09b9877be63d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:39.633244 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:39.634106 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:39.634537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:39.634987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] PATH_INFO: `/v3/users/2ef546ba84fa46d581ed6c82ec302e7d` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:39.636301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef546ba84fa46d581ed6c82ec302e7d)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:39.638497 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:39.697693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef546ba84fa46d581ed6c82ec302e7d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:39.707397 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef546ba84fa46d581ed6c82ec302e7d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:39.729789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef546ba84fa46d581ed6c82ec302e7d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:39.739399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef546ba84fa46d581ed6c82ec302e7d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:39.740544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3c364014-8a96-412f-87fe-788368c50cc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef546ba84fa46d581ed6c82ec302e7d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:39.756446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1324/2637] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:11:39 2022] DELETE /identity/v3/users/2ef546ba84fa46d581ed6c82ec302e7d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:39.950810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc422a8-aa82-4db2-b284-ede373a73bae service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:39.963286 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc422a8-aa82-4db2-b284-ede373a73bae service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.036471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.037659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.038104 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.038370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] PATH_INFO: `/v3/projects/20760466eac94b87b14c279bba223b2d` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.039525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20760466eac94b87b14c279bba223b2d)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.041421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.093715 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20760466eac94b87b14c279bba223b2d', 'request_id': 'req-ee105f80-2a59-4548-9d3e-d2bf77e56109'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:40.097079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:40.106223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20760466eac94b87b14c279bba223b2d', 'request_id': 'req-ee105f80-2a59-4548-9d3e-d2bf77e56109'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:40.117591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20760466eac94b87b14c279bba223b2d', 'request_id': 'req-ee105f80-2a59-4548-9d3e-d2bf77e56109'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:40.132218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] The token cache is being invalidate because project 20760466eac94b87b14c279bba223b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:11:40.132502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:40.133166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ee105f80-2a59-4548-9d3e-d2bf77e56109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:40.134985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1314/2638] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:11:39 2022] DELETE /identity/v3/projects/20760466eac94b87b14c279bba223b2d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:40.139735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926dbfe-12f1-4306-80ce-75c6e5e91e52 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.140024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926dbfe-12f1-4306-80ce-75c6e5e91e52 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.140460 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926dbfe-12f1-4306-80ce-75c6e5e91e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.193529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d926dbfe-12f1-4306-80ce-75c6e5e91e52 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:40.195067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d926dbfe-12f1-4306-80ce-75c6e5e91e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:40.256132 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1325/2639] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:11:40.263826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59f476-c4bf-4801-8a42-9668519cd18e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.264106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59f476-c4bf-4801-8a42-9668519cd18e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.264379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59f476-c4bf-4801-8a42-9668519cd18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.284117 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-7b59f476-c4bf-4801-8a42-9668519cd18e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:40.285303 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7b59f476-c4bf-4801-8a42-9668519cd18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:40.293195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1315/2640] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:40.297633 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926dbfe-12f1-4306-80ce-75c6e5e91e52 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.309522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926dbfe-12f1-4306-80ce-75c6e5e91e52 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.315969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a10883-670c-4edf-9f10-331e3edf4e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.316687 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a10883-670c-4edf-9f10-331e3edf4e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.317056 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a10883-670c-4edf-9f10-331e3edf4e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.317514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a10883-670c-4edf-9f10-331e3edf4e28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.318360 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a10883-670c-4edf-9f10-331e3edf4e28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:11:40.319007 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a10883-670c-4edf-9f10-331e3edf4e28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.320728 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a10883-670c-4edf-9f10-331e3edf4e28 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.341842 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59f476-c4bf-4801-8a42-9668519cd18e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.347628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1326/2641] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:11:40.353348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59f476-c4bf-4801-8a42-9668519cd18e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.358660 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4360df-176e-4b61-8a0d-1f089ef9fafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.359497 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4360df-176e-4b61-8a0d-1f089ef9fafc admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.359951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4360df-176e-4b61-8a0d-1f089ef9fafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.360527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4360df-176e-4b61-8a0d-1f089ef9fafc admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.363278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4360df-176e-4b61-8a0d-1f089ef9fafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.365240 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4360df-176e-4b61-8a0d-1f089ef9fafc admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.404018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a10883-670c-4edf-9f10-331e3edf4e28 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.411386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1316/2642] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:11:40 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:11:40.416000 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a10883-670c-4edf-9f10-331e3edf4e28 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.451611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:40.487644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bf7cc-8443-4681-bc2f-6d140e3eda35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.488257 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bf7cc-8443-4681-bc2f-6d140e3eda35 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.488592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bf7cc-8443-4681-bc2f-6d140e3eda35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.488870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bf7cc-8443-4681-bc2f-6d140e3eda35 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.489494 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bf7cc-8443-4681-bc2f-6d140e3eda35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.491539 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bf7cc-8443-4681-bc2f-6d140e3eda35 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.521959 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1327/2643] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:11:40 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:40.526251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4360df-176e-4b61-8a0d-1f089ef9fafc admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.536935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4360df-176e-4b61-8a0d-1f089ef9fafc admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.542114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ed9bc-19c6-4282-a8c5-e38d10fc7f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.542712 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ed9bc-19c6-4282-a8c5-e38d10fc7f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.543028 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ed9bc-19c6-4282-a8c5-e38d10fc7f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.543304 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ed9bc-19c6-4282-a8c5-e38d10fc7f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.543934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ed9bc-19c6-4282-a8c5-e38d10fc7f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.545555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ed9bc-19c6-4282-a8c5-e38d10fc7f56 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.557093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1317/2644] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:40.562185 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bf7cc-8443-4681-bc2f-6d140e3eda35 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.572741 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bf7cc-8443-4681-bc2f-6d140e3eda35 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.578559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dd466-b651-419a-ae7a-5a81ee78d7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.579427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dd466-b651-419a-ae7a-5a81ee78d7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.579703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dd466-b651-419a-ae7a-5a81ee78d7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.579973 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dd466-b651-419a-ae7a-5a81ee78d7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.580660 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dd466-b651-419a-ae7a-5a81ee78d7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.582359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dd466-b651-419a-ae7a-5a81ee78d7b2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.598489 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1328/2645] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:11:40.599596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ed9bc-19c6-4282-a8c5-e38d10fc7f56 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.611278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ed9bc-19c6-4282-a8c5-e38d10fc7f56 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.616522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ba602-aceb-40ec-ad70-2a061172584f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.617183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ba602-aceb-40ec-ad70-2a061172584f admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.617460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ba602-aceb-40ec-ad70-2a061172584f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.617733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ba602-aceb-40ec-ad70-2a061172584f admin admin] PATH_INFO: `/v3/projects/c010b6b6f33c44c48794fdd868ad1288/users/82f37a48a71b41fdb0739ac042c7af84/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.633881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ba602-aceb-40ec-ad70-2a061172584f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c010b6b6f33c44c48794fdd868ad1288, user_id=82f37a48a71b41fdb0739ac042c7af84, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.636573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ba602-aceb-40ec-ad70-2a061172584f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.650062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1318/2646] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:40 2022] PUT /identity/v3/projects/c010b6b6f33c44c48794fdd868ad1288/users/82f37a48a71b41fdb0739ac042c7af84/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:11:40.661441 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51aaa3-e7f3-46d5-b824-500add3efb35 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.661724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51aaa3-e7f3-46d5-b824-500add3efb35 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.661997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51aaa3-e7f3-46d5-b824-500add3efb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.688493 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9f51aaa3-e7f3-46d5-b824-500add3efb35 None None] MFA Rules not processed for user `82f37a48a71b41fdb0739ac042c7af84`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:40.690213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f51aaa3-e7f3-46d5-b824-500add3efb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:40.753074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1329/2647] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:40.757699 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ba602-aceb-40ec-ad70-2a061172584f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.816935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ba602-aceb-40ec-ad70-2a061172584f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.821432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a239e-dd23-48de-ad01-3ac63ba63f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.822053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a239e-dd23-48de-ad01-3ac63ba63f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.822324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a239e-dd23-48de-ad01-3ac63ba63f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.822588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a239e-dd23-48de-ad01-3ac63ba63f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.823230 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a239e-dd23-48de-ad01-3ac63ba63f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.825090 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a239e-dd23-48de-ad01-3ac63ba63f53 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.850318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1319/2648] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:11:40 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:40.854554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51aaa3-e7f3-46d5-b824-500add3efb35 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.865772 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51aaa3-e7f3-46d5-b824-500add3efb35 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.921784 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751910d2-4953-4291-ad62-bd1c49cda241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.922374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751910d2-4953-4291-ad62-bd1c49cda241 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.922650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751910d2-4953-4291-ad62-bd1c49cda241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.922920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751910d2-4953-4291-ad62-bd1c49cda241 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.923568 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751910d2-4953-4291-ad62-bd1c49cda241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.925507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751910d2-4953-4291-ad62-bd1c49cda241 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.948387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1330/2649] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:11:40 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:40.949507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a239e-dd23-48de-ad01-3ac63ba63f53 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.960245 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a239e-dd23-48de-ad01-3ac63ba63f53 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.964832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d8dd5-abf7-4f45-9a16-ff92652cefc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.965455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d8dd5-abf7-4f45-9a16-ff92652cefc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.965735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d8dd5-abf7-4f45-9a16-ff92652cefc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.966001 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d8dd5-abf7-4f45-9a16-ff92652cefc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:40.966629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d8dd5-abf7-4f45-9a16-ff92652cefc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:40.968244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d8dd5-abf7-4f45-9a16-ff92652cefc4 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:40.978874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1320/2650] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:40.982813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751910d2-4953-4291-ad62-bd1c49cda241 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:40.992536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751910d2-4953-4291-ad62-bd1c49cda241 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:40.997162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5bd04-f029-4938-a700-74c3607024d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:40.997802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5bd04-f029-4938-a700-74c3607024d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:40.998082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5bd04-f029-4938-a700-74c3607024d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:40.998353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5bd04-f029-4938-a700-74c3607024d8 admin admin] PATH_INFO: `/v3/projects/f13318b41b054c1d9c1e0dae972f5418/users/550616e9e3ad44d9bb2e898e016aa124/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:41.012944 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5bd04-f029-4938-a700-74c3607024d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13318b41b054c1d9c1e0dae972f5418, user_id=550616e9e3ad44d9bb2e898e016aa124, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:41.015621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5bd04-f029-4938-a700-74c3607024d8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:41.028252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1331/2651] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:40 2022] PUT /identity/v3/projects/f13318b41b054c1d9c1e0dae972f5418/users/550616e9e3ad44d9bb2e898e016aa124/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:41.032100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d8dd5-abf7-4f45-9a16-ff92652cefc4 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:41.042515 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d8dd5-abf7-4f45-9a16-ff92652cefc4 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:41.087776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:41.099107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf77bf0-b266-48d5-8257-3b0258bfe2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:41.099697 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf77bf0-b266-48d5-8257-3b0258bfe2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:41.099971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf77bf0-b266-48d5-8257-3b0258bfe2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:41.100238 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf77bf0-b266-48d5-8257-3b0258bfe2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:41.100858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf77bf0-b266-48d5-8257-3b0258bfe2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:41.102422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf77bf0-b266-48d5-8257-3b0258bfe2f8 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:41.111724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1321/2652] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:41.115783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5bd04-f029-4938-a700-74c3607024d8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:41.126308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5bd04-f029-4938-a700-74c3607024d8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:41.130932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c43a10-6d07-4cfd-b7a6-794e99dde3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:41.131608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c43a10-6d07-4cfd-b7a6-794e99dde3c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:41.131889 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c43a10-6d07-4cfd-b7a6-794e99dde3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:41.132165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c43a10-6d07-4cfd-b7a6-794e99dde3c0 admin admin] PATH_INFO: `/v3/projects/f13318b41b054c1d9c1e0dae972f5418/users/550616e9e3ad44d9bb2e898e016aa124/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:41.134075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c43a10-6d07-4cfd-b7a6-794e99dde3c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13318b41b054c1d9c1e0dae972f5418, user_id=550616e9e3ad44d9bb2e898e016aa124, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:41.136779 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c43a10-6d07-4cfd-b7a6-794e99dde3c0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:41.139819 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:41.149213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1332/2653] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:41 2022] PUT /identity/v3/projects/f13318b41b054c1d9c1e0dae972f5418/users/550616e9e3ad44d9bb2e898e016aa124/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:11:41.159860 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef7669-8bb1-48d4-840a-8a5e06fb6181 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:41.160140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef7669-8bb1-48d4-840a-8a5e06fb6181 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:41.160406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef7669-8bb1-48d4-840a-8a5e06fb6181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:41.187086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-d2ef7669-8bb1-48d4-840a-8a5e06fb6181 None None] MFA Rules not processed for user `550616e9e3ad44d9bb2e898e016aa124`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:41.188273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d2ef7669-8bb1-48d4-840a-8a5e06fb6181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:41.260528 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1322/2654] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:41.272284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c43a10-6d07-4cfd-b7a6-794e99dde3c0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:41.274295 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-89c43a10-6d07-4cfd-b7a6-794e99dde3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:41.354230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c43a10-6d07-4cfd-b7a6-794e99dde3c0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:41.357849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53dee1-98ec-4858-b3e7-f778079b0bc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:41.358486 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53dee1-98ec-4858-b3e7-f778079b0bc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:41.358766 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53dee1-98ec-4858-b3e7-f778079b0bc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:41.359085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53dee1-98ec-4858-b3e7-f778079b0bc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:41.367845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53dee1-98ec-4858-b3e7-f778079b0bc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:41.369259 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53dee1-98ec-4858-b3e7-f778079b0bc7 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:41.379272 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1333/2655] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:41.800298 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef7669-8bb1-48d4-840a-8a5e06fb6181 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:41.862274 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef7669-8bb1-48d4-840a-8a5e06fb6181 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:41.866580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:41.867218 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:41.867491 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:41.867756 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] PATH_INFO: `/v3/users/82f37a48a71b41fdb0739ac042c7af84` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:41.868829 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f37a48a71b41fdb0739ac042c7af84)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:41.870717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:41.915415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f37a48a71b41fdb0739ac042c7af84'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:41.922349 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f37a48a71b41fdb0739ac042c7af84'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:41.928979 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f37a48a71b41fdb0739ac042c7af84'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:41.936790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:41.942156 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f37a48a71b41fdb0739ac042c7af84'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:41.942969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f37a48a71b41fdb0739ac042c7af84'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:41.949934 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1323/2656] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:11:41 2022] DELETE /identity/v3/users/82f37a48a71b41fdb0739ac042c7af84 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:41.954311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1dce5-6fa0-46a2-836b-2b064529bc7b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:41.954832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1dce5-6fa0-46a2-836b-2b064529bc7b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:41.954832 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1dce5-6fa0-46a2-836b-2b064529bc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:41.974496 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5ff1dce5-6fa0-46a2-836b-2b064529bc7b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:41.975856 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5ff1dce5-6fa0-46a2-836b-2b064529bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:42.035693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1334/2657] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:42.045494 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:42.047023 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:42.106557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58b35d-f781-433e-97b7-6096530b5fe8 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:42.110429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f608ae6-f94d-4ee9-9a5a-6079ba14aee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:42.111081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f608ae6-f94d-4ee9-9a5a-6079ba14aee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:42.111364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f608ae6-f94d-4ee9-9a5a-6079ba14aee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:42.111737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f608ae6-f94d-4ee9-9a5a-6079ba14aee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:42.120741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f608ae6-f94d-4ee9-9a5a-6079ba14aee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:42.122127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f608ae6-f94d-4ee9-9a5a-6079ba14aee4 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:42.132171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1324/2658] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:42.399983 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1dce5-6fa0-46a2-836b-2b064529bc7b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:42.401905 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5ff1dce5-6fa0-46a2-836b-2b064529bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:42.419344 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1dce5-6fa0-46a2-836b-2b064529bc7b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:42.424510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:42.425138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:42.425418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:42.425667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] PATH_INFO: `/v3/projects/c010b6b6f33c44c48794fdd868ad1288` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:42.426782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c010b6b6f33c44c48794fdd868ad1288)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:42.428756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:42.475689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c010b6b6f33c44c48794fdd868ad1288', 'request_id': 'req-861f6163-ffe9-4d04-b991-93e950322b2b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.483029 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c010b6b6f33c44c48794fdd868ad1288', 'request_id': 'req-861f6163-ffe9-4d04-b991-93e950322b2b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.489631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c010b6b6f33c44c48794fdd868ad1288', 'request_id': 'req-861f6163-ffe9-4d04-b991-93e950322b2b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.499901 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] The token cache is being invalidate because project c010b6b6f33c44c48794fdd868ad1288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:11:42.500188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.501079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.503197 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1335/2659] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:11:42 2022] DELETE /identity/v3/projects/c010b6b6f33c44c48794fdd868ad1288 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:42.507116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f608ae6-f94d-4ee9-9a5a-6079ba14aee4 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:42.516391 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f608ae6-f94d-4ee9-9a5a-6079ba14aee4 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:42.573427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:42.574047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:42.574320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:42.574587 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] PATH_INFO: `/v3/users/550616e9e3ad44d9bb2e898e016aa124` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:42.575711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=550616e9e3ad44d9bb2e898e016aa124)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:42.577629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:42.621832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550616e9e3ad44d9bb2e898e016aa124'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.629200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550616e9e3ad44d9bb2e898e016aa124'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.635810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550616e9e3ad44d9bb2e898e016aa124'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.647790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550616e9e3ad44d9bb2e898e016aa124'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.648354 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f1920526-cd96-4413-9b48-ed893b2703dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550616e9e3ad44d9bb2e898e016aa124'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:42.653964 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1325/2660] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:11:42 2022] DELETE /identity/v3/users/550616e9e3ad44d9bb2e898e016aa124 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:42.930897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:42.932636 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:42.940811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:42.947984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f6163-ffe9-4d04-b991-93e950322b2b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.009271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.010024 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.010169 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.010439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] PATH_INFO: `/v3/projects/f13318b41b054c1d9c1e0dae972f5418` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.011563 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f13318b41b054c1d9c1e0dae972f5418)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.013455 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.061302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f13318b41b054c1d9c1e0dae972f5418', 'request_id': 'req-f193373f-cffb-40dd-8dcc-09ae9bc95968'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:43.068673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f13318b41b054c1d9c1e0dae972f5418', 'request_id': 'req-f193373f-cffb-40dd-8dcc-09ae9bc95968'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:43.075659 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f13318b41b054c1d9c1e0dae972f5418', 'request_id': 'req-f193373f-cffb-40dd-8dcc-09ae9bc95968'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:43.087155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] The token cache is being invalidate because project f13318b41b054c1d9c1e0dae972f5418 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:11:43.087647 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:43.088297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f193373f-cffb-40dd-8dcc-09ae9bc95968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:43.089839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1336/2661] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:11:42 2022] DELETE /identity/v3/projects/f13318b41b054c1d9c1e0dae972f5418 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:43.094754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593757cc-a4f7-4d24-83b5-2a4133808eb0 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.095267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593757cc-a4f7-4d24-83b5-2a4133808eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.095381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593757cc-a4f7-4d24-83b5-2a4133808eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.116171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-593757cc-a4f7-4d24-83b5-2a4133808eb0 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:43.117362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-593757cc-a4f7-4d24-83b5-2a4133808eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:43.125966 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:43.186927 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1326/2662] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:11:43.194850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9493828-d660-4359-9728-1053eb5ac8b9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.195168 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9493828-d660-4359-9728-1053eb5ac8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.195447 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9493828-d660-4359-9728-1053eb5ac8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.215546 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e9493828-d660-4359-9728-1053eb5ac8b9 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:43.216774 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e9493828-d660-4359-9728-1053eb5ac8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:43.225414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1337/2663] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:43.230293 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593757cc-a4f7-4d24-83b5-2a4133808eb0 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.244821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593757cc-a4f7-4d24-83b5-2a4133808eb0 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.249670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f938b20-587d-4eec-b184-35a4d6f883c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.250269 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f938b20-587d-4eec-b184-35a4d6f883c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.250795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f938b20-587d-4eec-b184-35a4d6f883c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.251161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f938b20-587d-4eec-b184-35a4d6f883c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.251799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f938b20-587d-4eec-b184-35a4d6f883c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:11:43.252655 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f938b20-587d-4eec-b184-35a4d6f883c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.254226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f938b20-587d-4eec-b184-35a4d6f883c3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.266185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1327/2664] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:11:43.270639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9493828-d660-4359-9728-1053eb5ac8b9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.280273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9493828-d660-4359-9728-1053eb5ac8b9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.285104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331c458-647a-4a39-aa15-43825fba8c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.285708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331c458-647a-4a39-aa15-43825fba8c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.286228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331c458-647a-4a39-aa15-43825fba8c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.286228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331c458-647a-4a39-aa15-43825fba8c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.286850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331c458-647a-4a39-aa15-43825fba8c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.288778 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331c458-647a-4a39-aa15-43825fba8c4b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.321233 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1338/2665] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:11:43 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:43.322742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f938b20-587d-4eec-b184-35a4d6f883c3 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.333037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f938b20-587d-4eec-b184-35a4d6f883c3 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.388923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211e638-732b-4687-9132-96a5ce56ed98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.389501 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211e638-732b-4687-9132-96a5ce56ed98 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.390301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211e638-732b-4687-9132-96a5ce56ed98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.390301 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211e638-732b-4687-9132-96a5ce56ed98 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.390751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211e638-732b-4687-9132-96a5ce56ed98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.392779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211e638-732b-4687-9132-96a5ce56ed98 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.414889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1328/2666] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:11:43 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:11:43.419036 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331c458-647a-4a39-aa15-43825fba8c4b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.429855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331c458-647a-4a39-aa15-43825fba8c4b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.434619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcff8b5-93f0-4f63-956a-3284df35810e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.435303 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff8b5-93f0-4f63-956a-3284df35810e admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.435645 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff8b5-93f0-4f63-956a-3284df35810e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.435987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcff8b5-93f0-4f63-956a-3284df35810e admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.436727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcff8b5-93f0-4f63-956a-3284df35810e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.438349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcff8b5-93f0-4f63-956a-3284df35810e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.448631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1339/2667] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:11:43.452550 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211e638-732b-4687-9132-96a5ce56ed98 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.462493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211e638-732b-4687-9132-96a5ce56ed98 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.467226 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc7ffa-ff63-4d6a-99be-06a7a0b97c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.467827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc7ffa-ff63-4d6a-99be-06a7a0b97c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.468105 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc7ffa-ff63-4d6a-99be-06a7a0b97c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.468376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc7ffa-ff63-4d6a-99be-06a7a0b97c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.469019 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc7ffa-ff63-4d6a-99be-06a7a0b97c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.470656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc7ffa-ff63-4d6a-99be-06a7a0b97c35 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.482220 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1329/2668] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:43.486120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcff8b5-93f0-4f63-956a-3284df35810e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.497041 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcff8b5-93f0-4f63-956a-3284df35810e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.502209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf2521-6f18-4b16-8735-8816a241cf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.506432 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf2521-6f18-4b16-8735-8816a241cf90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.506785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf2521-6f18-4b16-8735-8816a241cf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.507160 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf2521-6f18-4b16-8735-8816a241cf90 admin admin] PATH_INFO: `/v3/projects/7effebdc16bc4ffd936ae0c5af770062/users/5df32441c9464006a5c6200bd9f2e31e/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.523172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf2521-6f18-4b16-8735-8816a241cf90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7effebdc16bc4ffd936ae0c5af770062, user_id=5df32441c9464006a5c6200bd9f2e31e, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.526070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf2521-6f18-4b16-8735-8816a241cf90 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.540082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1340/2669] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:43 2022] PUT /identity/v3/projects/7effebdc16bc4ffd936ae0c5af770062/users/5df32441c9464006a5c6200bd9f2e31e/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:43.552151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a5338-c885-4b88-a9f1-0177d44f5778 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.552757 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a5338-c885-4b88-a9f1-0177d44f5778 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.553306 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a5338-c885-4b88-a9f1-0177d44f5778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.584049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-f06a5338-c885-4b88-a9f1-0177d44f5778 None None] MFA Rules not processed for user `5df32441c9464006a5c6200bd9f2e31e`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:43.585138 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f06a5338-c885-4b88-a9f1-0177d44f5778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:43.653392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1330/2670] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:11:43.657854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf2521-6f18-4b16-8735-8816a241cf90 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.690433 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:43.724252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf2521-6f18-4b16-8735-8816a241cf90 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.729123 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429da5d-2a4d-4faf-a145-a37b79317ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.730201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429da5d-2a4d-4faf-a145-a37b79317ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.730201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429da5d-2a4d-4faf-a145-a37b79317ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.730409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429da5d-2a4d-4faf-a145-a37b79317ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.730866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429da5d-2a4d-4faf-a145-a37b79317ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.733201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429da5d-2a4d-4faf-a145-a37b79317ca7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.760673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1341/2671] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:11:43 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:43.764726 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a5338-c885-4b88-a9f1-0177d44f5778 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.775535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a5338-c885-4b88-a9f1-0177d44f5778 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.780381 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:43.834114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718d8a2-100f-4471-92a5-f4e06754cd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.834879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718d8a2-100f-4471-92a5-f4e06754cd0d admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.835283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718d8a2-100f-4471-92a5-f4e06754cd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.835648 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718d8a2-100f-4471-92a5-f4e06754cd0d admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.836388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718d8a2-100f-4471-92a5-f4e06754cd0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.838384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718d8a2-100f-4471-92a5-f4e06754cd0d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.860229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1331/2672] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:11:43 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:11:43.864787 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429da5d-2a4d-4faf-a145-a37b79317ca7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.874848 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429da5d-2a4d-4faf-a145-a37b79317ca7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.881225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba1c93-a56c-44c0-bab1-c5114cf0e3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.881897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba1c93-a56c-44c0-bab1-c5114cf0e3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.882246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba1c93-a56c-44c0-bab1-c5114cf0e3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.882577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba1c93-a56c-44c0-bab1-c5114cf0e3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.883334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba1c93-a56c-44c0-bab1-c5114cf0e3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.884987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba1c93-a56c-44c0-bab1-c5114cf0e3e1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.900103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1342/2673] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:43.901357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718d8a2-100f-4471-92a5-f4e06754cd0d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.911489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718d8a2-100f-4471-92a5-f4e06754cd0d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:43.916055 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa9b51-017a-43c9-ac6d-5c4ebf333a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:43.916705 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa9b51-017a-43c9-ac6d-5c4ebf333a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:43.916984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa9b51-017a-43c9-ac6d-5c4ebf333a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:43.917260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa9b51-017a-43c9-ac6d-5c4ebf333a13 admin admin] PATH_INFO: `/v3/projects/b90f83d142ee4f7c8d9b9f3983c83892/users/fd56ba5ae31f4e88873543c71fe55c40/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:43.932560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa9b51-017a-43c9-ac6d-5c4ebf333a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90f83d142ee4f7c8d9b9f3983c83892, user_id=fd56ba5ae31f4e88873543c71fe55c40, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:43.935216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa9b51-017a-43c9-ac6d-5c4ebf333a13 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:43.949140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1332/2674] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:43 2022] PUT /identity/v3/projects/b90f83d142ee4f7c8d9b9f3983c83892/users/fd56ba5ae31f4e88873543c71fe55c40/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:11:43.953058 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba1c93-a56c-44c0-bab1-c5114cf0e3e1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:43.963643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba1c93-a56c-44c0-bab1-c5114cf0e3e1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:44.022837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e449b5f-a52e-41f1-92fb-12ed867a9a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:44.023561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e449b5f-a52e-41f1-92fb-12ed867a9a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:44.023742 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e449b5f-a52e-41f1-92fb-12ed867a9a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:44.024013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e449b5f-a52e-41f1-92fb-12ed867a9a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:44.024656 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e449b5f-a52e-41f1-92fb-12ed867a9a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:44.026243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e449b5f-a52e-41f1-92fb-12ed867a9a72 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:44.036084 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1343/2675] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:11:44.040340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa9b51-017a-43c9-ac6d-5c4ebf333a13 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:44.050935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa9b51-017a-43c9-ac6d-5c4ebf333a13 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:44.055436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897af63-3de2-4323-a218-5176db8e1c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:44.056102 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897af63-3de2-4323-a218-5176db8e1c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:44.056376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897af63-3de2-4323-a218-5176db8e1c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:44.056647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897af63-3de2-4323-a218-5176db8e1c43 admin admin] PATH_INFO: `/v3/projects/b90f83d142ee4f7c8d9b9f3983c83892/users/fd56ba5ae31f4e88873543c71fe55c40/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:44.058578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897af63-3de2-4323-a218-5176db8e1c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90f83d142ee4f7c8d9b9f3983c83892, user_id=fd56ba5ae31f4e88873543c71fe55c40, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:44.061187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897af63-3de2-4323-a218-5176db8e1c43 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:44.071718 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1333/2676] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:44 2022] PUT /identity/v3/projects/b90f83d142ee4f7c8d9b9f3983c83892/users/fd56ba5ae31f4e88873543c71fe55c40/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:44.082105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135731bf-1927-4b9f-abc4-2af018302754 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:44.082395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135731bf-1927-4b9f-abc4-2af018302754 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:44.082673 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135731bf-1927-4b9f-abc4-2af018302754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:44.110964 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-135731bf-1927-4b9f-abc4-2af018302754 None None] MFA Rules not processed for user `fd56ba5ae31f4e88873543c71fe55c40`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:44.112731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-135731bf-1927-4b9f-abc4-2af018302754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:44.183960 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1344/2677] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:11:44.194653 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897af63-3de2-4323-a218-5176db8e1c43 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:44.196409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8897af63-3de2-4323-a218-5176db8e1c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:44.258561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897af63-3de2-4323-a218-5176db8e1c43 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:44.262183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e6be8-49fc-4135-9325-6887853064d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:44.262777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e6be8-49fc-4135-9325-6887853064d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:44.263083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e6be8-49fc-4135-9325-6887853064d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:44.263355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e6be8-49fc-4135-9325-6887853064d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:44.273246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e6be8-49fc-4135-9325-6887853064d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:44.274663 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e6be8-49fc-4135-9325-6887853064d4 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:44.285097 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1334/2678] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:45.736306 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135731bf-1927-4b9f-abc4-2af018302754 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:45.764050 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:45.801995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135731bf-1927-4b9f-abc4-2af018302754 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:45.806749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:45.807408 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:45.807692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:45.807968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] PATH_INFO: `/v3/users/5df32441c9464006a5c6200bd9f2e31e` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:45.809098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df32441c9464006a5c6200bd9f2e31e)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:45.811102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:45.861404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df32441c9464006a5c6200bd9f2e31e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:45.868314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df32441c9464006a5c6200bd9f2e31e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:45.880729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df32441c9464006a5c6200bd9f2e31e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:45.881400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df32441c9464006a5c6200bd9f2e31e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:45.882018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df32441c9464006a5c6200bd9f2e31e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:45.895629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1345/2679] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:11:45 2022] DELETE /identity/v3/users/5df32441c9464006a5c6200bd9f2e31e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:45.900395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623a73d-4d14-44ee-8641-f3ca95fc850f None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:45.900869 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623a73d-4d14-44ee-8641-f3ca95fc850f None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:45.900985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623a73d-4d14-44ee-8641-f3ca95fc850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:45.918893 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-3623a73d-4d14-44ee-8641-f3ca95fc850f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:45.920100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3623a73d-4d14-44ee-8641-f3ca95fc850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:45.977588 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1335/2680] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:45.987484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:45.988872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:46.044981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd1a82-fc85-482a-89f2-ff4038bec63f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:46.048651 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd32d-2419-46b6-8c1d-3f81c068e400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:46.049735 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd32d-2419-46b6-8c1d-3f81c068e400 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:46.050013 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd32d-2419-46b6-8c1d-3f81c068e400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:46.050282 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd32d-2419-46b6-8c1d-3f81c068e400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:46.059025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cd32d-2419-46b6-8c1d-3f81c068e400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:46.060442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cd32d-2419-46b6-8c1d-3f81c068e400 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:46.070203 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1346/2681] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:46.341916 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623a73d-4d14-44ee-8641-f3ca95fc850f None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:46.343345 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3623a73d-4d14-44ee-8641-f3ca95fc850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:46.351524 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:46.355754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623a73d-4d14-44ee-8641-f3ca95fc850f None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:46.361839 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:46.362594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:46.362866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:46.363159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] PATH_INFO: `/v3/projects/7effebdc16bc4ffd936ae0c5af770062` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:46.364272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7effebdc16bc4ffd936ae0c5af770062)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:46.366146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:46.417578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7effebdc16bc4ffd936ae0c5af770062', 'request_id': 'req-d6880a03-f41a-437b-ac6d-28eccc61cbb7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.423553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7effebdc16bc4ffd936ae0c5af770062', 'request_id': 'req-d6880a03-f41a-437b-ac6d-28eccc61cbb7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.430880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7effebdc16bc4ffd936ae0c5af770062', 'request_id': 'req-d6880a03-f41a-437b-ac6d-28eccc61cbb7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.441045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] The token cache is being invalidate because project 7effebdc16bc4ffd936ae0c5af770062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:11:46.441330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.441937 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.443430 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1336/2682] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:11:46 2022] DELETE /identity/v3/projects/7effebdc16bc4ffd936ae0c5af770062 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:46.447342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd32d-2419-46b6-8c1d-3f81c068e400 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:46.456648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd32d-2419-46b6-8c1d-3f81c068e400 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:46.498720 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:46.512262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:46.512864 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:46.513138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:46.513406 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] PATH_INFO: `/v3/users/fd56ba5ae31f4e88873543c71fe55c40` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:46.514499 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd56ba5ae31f4e88873543c71fe55c40)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:46.516422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:46.561214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd56ba5ae31f4e88873543c71fe55c40'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.567879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd56ba5ae31f4e88873543c71fe55c40'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.580922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd56ba5ae31f4e88873543c71fe55c40'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.580922 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd56ba5ae31f4e88873543c71fe55c40'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.581147 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cb7a697c-238a-4ab7-a52f-db518104f77f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd56ba5ae31f4e88873543c71fe55c40'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:46.593118 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1347/2683] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:11:46 2022] DELETE /identity/v3/users/fd56ba5ae31f4e88873543c71fe55c40 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:46.868288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:46.869684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:46.886629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6880a03-f41a-437b-ac6d-28eccc61cbb7 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:46.946691 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:46.947437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:46.947610 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:46.947879 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] PATH_INFO: `/v3/projects/b90f83d142ee4f7c8d9b9f3983c83892` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:46.948986 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b90f83d142ee4f7c8d9b9f3983c83892)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:46.950928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:46.994936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b90f83d142ee4f7c8d9b9f3983c83892', 'request_id': 'req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:47.001603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b90f83d142ee4f7c8d9b9f3983c83892', 'request_id': 'req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:47.009362 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b90f83d142ee4f7c8d9b9f3983c83892', 'request_id': 'req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:47.019423 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] The token cache is being invalidate because project b90f83d142ee4f7c8d9b9f3983c83892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:11:47.019713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:47.020341 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-ce4c7b3b-6ea9-4815-8f35-8810802fb6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:47.021787 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1337/2684] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:11:46 2022] DELETE /identity/v3/projects/b90f83d142ee4f7c8d9b9f3983c83892 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:47.026718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f83dee-7623-4632-861a-91e82707e471 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.027216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f83dee-7623-4632-861a-91e82707e471 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.027323 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f83dee-7623-4632-861a-91e82707e471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.046168 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e5f83dee-7623-4632-861a-91e82707e471 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:47.047400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e5f83dee-7623-4632-861a-91e82707e471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:47.105094 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1348/2685] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:47.114786 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d2dca-a7fc-4b6f-9d79-e1cc82dcc49a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.115028 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d2dca-a7fc-4b6f-9d79-e1cc82dcc49a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.115296 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d2dca-a7fc-4b6f-9d79-e1cc82dcc49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.135029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-996d2dca-a7fc-4b6f-9d79-e1cc82dcc49a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:47.136193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-996d2dca-a7fc-4b6f-9d79-e1cc82dcc49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:47.145416 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1338/2686] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:47.149606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f83dee-7623-4632-861a-91e82707e471 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.159763 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f83dee-7623-4632-861a-91e82707e471 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.164289 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ff8a5-a1bc-4f48-8d92-7b1e84d9cdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.165139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ff8a5-a1bc-4f48-8d92-7b1e84d9cdab admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.165422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ff8a5-a1bc-4f48-8d92-7b1e84d9cdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.165691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ff8a5-a1bc-4f48-8d92-7b1e84d9cdab admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.166327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ff8a5-a1bc-4f48-8d92-7b1e84d9cdab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:11:47.166622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ff8a5-a1bc-4f48-8d92-7b1e84d9cdab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.168230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ff8a5-a1bc-4f48-8d92-7b1e84d9cdab admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.180126 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1349/2687] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:11:47.184178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d2dca-a7fc-4b6f-9d79-e1cc82dcc49a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.193163 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d2dca-a7fc-4b6f-9d79-e1cc82dcc49a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.197795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60ce76-2fb2-43c9-9b2d-336466aca612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.198584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60ce76-2fb2-43c9-9b2d-336466aca612 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.198861 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60ce76-2fb2-43c9-9b2d-336466aca612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.199169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60ce76-2fb2-43c9-9b2d-336466aca612 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.199798 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60ce76-2fb2-43c9-9b2d-336466aca612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.201670 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60ce76-2fb2-43c9-9b2d-336466aca612 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.210962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:47.230049 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1339/2688] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:11:47 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:47.234140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ff8a5-a1bc-4f48-8d92-7b1e84d9cdab admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.244746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ff8a5-a1bc-4f48-8d92-7b1e84d9cdab admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.293006 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:47.304017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea3456-f20b-4936-8157-c7e9404e3327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.304622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea3456-f20b-4936-8157-c7e9404e3327 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.304917 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea3456-f20b-4936-8157-c7e9404e3327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.305174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea3456-f20b-4936-8157-c7e9404e3327 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.305789 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea3456-f20b-4936-8157-c7e9404e3327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.307744 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea3456-f20b-4936-8157-c7e9404e3327 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.327866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1350/2689] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:11:47 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:47.331969 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60ce76-2fb2-43c9-9b2d-336466aca612 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.341965 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60ce76-2fb2-43c9-9b2d-336466aca612 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.347109 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1a461-f128-4ae1-9522-9f1123d94672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.347724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1a461-f128-4ae1-9522-9f1123d94672 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.347996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1a461-f128-4ae1-9522-9f1123d94672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.348262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1a461-f128-4ae1-9522-9f1123d94672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.348885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1a461-f128-4ae1-9522-9f1123d94672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.350445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1a461-f128-4ae1-9522-9f1123d94672 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.360288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1340/2690] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:47.364512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea3456-f20b-4936-8157-c7e9404e3327 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.374324 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea3456-f20b-4936-8157-c7e9404e3327 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.379116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ac18a-9e7d-40a8-92ca-8aa8f7f51e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.379703 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ac18a-9e7d-40a8-92ca-8aa8f7f51e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.379982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ac18a-9e7d-40a8-92ca-8aa8f7f51e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.380256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ac18a-9e7d-40a8-92ca-8aa8f7f51e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.380884 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ac18a-9e7d-40a8-92ca-8aa8f7f51e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.382451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ac18a-9e7d-40a8-92ca-8aa8f7f51e33 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.392767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1351/2691] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:47.396602 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1a461-f128-4ae1-9522-9f1123d94672 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.407103 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1a461-f128-4ae1-9522-9f1123d94672 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.413229 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d29f57-1842-45f5-a354-1dca4a65882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.413412 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d29f57-1842-45f5-a354-1dca4a65882c admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.413759 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d29f57-1842-45f5-a354-1dca4a65882c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.414038 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d29f57-1842-45f5-a354-1dca4a65882c admin admin] PATH_INFO: `/v3/projects/b79ef21804e94ba3b54458b2d7b2e21c/users/d3c6de97ae5f4949afff96fd712245e6/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.429555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d29f57-1842-45f5-a354-1dca4a65882c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79ef21804e94ba3b54458b2d7b2e21c, user_id=d3c6de97ae5f4949afff96fd712245e6, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.432225 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d29f57-1842-45f5-a354-1dca4a65882c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.444380 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1341/2692] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:47 2022] PUT /identity/v3/projects/b79ef21804e94ba3b54458b2d7b2e21c/users/d3c6de97ae5f4949afff96fd712245e6/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:47.454587 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342efecc-3557-412f-9659-39dc8e324a1e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.454892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342efecc-3557-412f-9659-39dc8e324a1e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.455198 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342efecc-3557-412f-9659-39dc8e324a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.482518 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-342efecc-3557-412f-9659-39dc8e324a1e None None] MFA Rules not processed for user `d3c6de97ae5f4949afff96fd712245e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:47.483762 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-342efecc-3557-412f-9659-39dc8e324a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:47.547201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1352/2693] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:47.551852 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d29f57-1842-45f5-a354-1dca4a65882c admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.613731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d29f57-1842-45f5-a354-1dca4a65882c admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.618360 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef47b42-c4fc-4e49-9a8e-14fe4ae4cc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.618970 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef47b42-c4fc-4e49-9a8e-14fe4ae4cc25 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.619253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef47b42-c4fc-4e49-9a8e-14fe4ae4cc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.619527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef47b42-c4fc-4e49-9a8e-14fe4ae4cc25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.620139 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef47b42-c4fc-4e49-9a8e-14fe4ae4cc25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.622016 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef47b42-c4fc-4e49-9a8e-14fe4ae4cc25 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.648029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1342/2694] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:11:47 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:47.652034 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342efecc-3557-412f-9659-39dc8e324a1e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.662634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342efecc-3557-412f-9659-39dc8e324a1e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.717033 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50e632-be24-4975-9371-85ba74179bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.717631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50e632-be24-4975-9371-85ba74179bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.717911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50e632-be24-4975-9371-85ba74179bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.718182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50e632-be24-4975-9371-85ba74179bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.718799 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50e632-be24-4975-9371-85ba74179bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.720736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50e632-be24-4975-9371-85ba74179bf0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.741258 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1353/2695] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:11:47 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:11:47.745219 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef47b42-c4fc-4e49-9a8e-14fe4ae4cc25 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.756189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef47b42-c4fc-4e49-9a8e-14fe4ae4cc25 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.760746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c25241-42f3-4e41-b34f-99411962ff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.761330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c25241-42f3-4e41-b34f-99411962ff5a admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.761605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c25241-42f3-4e41-b34f-99411962ff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.761875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c25241-42f3-4e41-b34f-99411962ff5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.762506 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c25241-42f3-4e41-b34f-99411962ff5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.764151 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c25241-42f3-4e41-b34f-99411962ff5a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.774266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1343/2696] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:47.778329 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50e632-be24-4975-9371-85ba74179bf0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.788226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50e632-be24-4975-9371-85ba74179bf0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.792814 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50558096-7f0c-4e41-92b3-5b91908a3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.793400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50558096-7f0c-4e41-92b3-5b91908a3183 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.793680 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50558096-7f0c-4e41-92b3-5b91908a3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.793953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50558096-7f0c-4e41-92b3-5b91908a3183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.794609 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50558096-7f0c-4e41-92b3-5b91908a3183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.796200 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50558096-7f0c-4e41-92b3-5b91908a3183 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.806283 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1354/2697] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:47.810192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c25241-42f3-4e41-b34f-99411962ff5a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:47.819906 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c25241-42f3-4e41-b34f-99411962ff5a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:47.824421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95592fa2-c21d-4f5f-8f4a-a13b9135fbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:47.825044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95592fa2-c21d-4f5f-8f4a-a13b9135fbbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.825318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95592fa2-c21d-4f5f-8f4a-a13b9135fbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.825584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95592fa2-c21d-4f5f-8f4a-a13b9135fbbe admin admin] PATH_INFO: `/v3/projects/7d5f5b21b51d44a19db73e5cb52b7804/users/6045b1bb79204607aeb7f9fb622be985/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.840159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95592fa2-c21d-4f5f-8f4a-a13b9135fbbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5f5b21b51d44a19db73e5cb52b7804, user_id=6045b1bb79204607aeb7f9fb622be985, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:47.842754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95592fa2-c21d-4f5f-8f4a-a13b9135fbbe admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:47.855839 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1344/2698] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:47 2022] PUT /identity/v3/projects/7d5f5b21b51d44a19db73e5cb52b7804/users/6045b1bb79204607aeb7f9fb622be985/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:47.865874 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dbcea-351f-442c-9f5d-a8075e2b4eba None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:47.866162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dbcea-351f-442c-9f5d-a8075e2b4eba None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:47.866442 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dbcea-351f-442c-9f5d-a8075e2b4eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:47.894243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-113dbcea-351f-442c-9f5d-a8075e2b4eba None None] MFA Rules not processed for user `6045b1bb79204607aeb7f9fb622be985`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:47.895529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-113dbcea-351f-442c-9f5d-a8075e2b4eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:47.925299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:47.964549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1355/2699] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:47.969079 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95592fa2-c21d-4f5f-8f4a-a13b9135fbbe admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:48.028376 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95592fa2-c21d-4f5f-8f4a-a13b9135fbbe admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:48.032972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b602db0-8bda-4357-a02f-54786f490d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:48.033561 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b602db0-8bda-4357-a02f-54786f490d7b admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:48.033832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b602db0-8bda-4357-a02f-54786f490d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:48.034099 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b602db0-8bda-4357-a02f-54786f490d7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:48.034711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b602db0-8bda-4357-a02f-54786f490d7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:48.036594 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b602db0-8bda-4357-a02f-54786f490d7b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:48.045762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:48.062571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1345/2700] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:11:47 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:48.067093 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dbcea-351f-442c-9f5d-a8075e2b4eba None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:48.077537 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dbcea-351f-442c-9f5d-a8075e2b4eba None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:48.131875 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59811cfe-442b-4911-8efb-382259d3eddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:48.132469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59811cfe-442b-4911-8efb-382259d3eddd admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:48.132745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59811cfe-442b-4911-8efb-382259d3eddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:48.133012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59811cfe-442b-4911-8efb-382259d3eddd admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:48.133623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59811cfe-442b-4911-8efb-382259d3eddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:48.135544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59811cfe-442b-4911-8efb-382259d3eddd admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:48.156590 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1356/2701] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:11:48 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:48.160527 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b602db0-8bda-4357-a02f-54786f490d7b admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:48.170395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b602db0-8bda-4357-a02f-54786f490d7b admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:48.174854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ff14a-98fe-44f4-86fb-5ce89b4ef2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:48.175518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ff14a-98fe-44f4-86fb-5ce89b4ef2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:48.175739 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ff14a-98fe-44f4-86fb-5ce89b4ef2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:48.176004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ff14a-98fe-44f4-86fb-5ce89b4ef2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:48.176626 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ff14a-98fe-44f4-86fb-5ce89b4ef2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:48.178231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ff14a-98fe-44f4-86fb-5ce89b4ef2d3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:48.187731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1346/2702] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:48.191971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59811cfe-442b-4911-8efb-382259d3eddd admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:48.201509 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59811cfe-442b-4911-8efb-382259d3eddd admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:48.206681 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d983a82-3399-4b24-8d78-a03f5bd78408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:48.207355 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d983a82-3399-4b24-8d78-a03f5bd78408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:48.207637 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d983a82-3399-4b24-8d78-a03f5bd78408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:48.207907 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d983a82-3399-4b24-8d78-a03f5bd78408 admin admin] PATH_INFO: `/v3/projects/3e77bc90040c4171b10d8a6e0dda9a03/users/b3bc736526ba4bfca203384235ddb37e/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:48.222536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d983a82-3399-4b24-8d78-a03f5bd78408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e77bc90040c4171b10d8a6e0dda9a03, user_id=b3bc736526ba4bfca203384235ddb37e, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:48.225187 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d983a82-3399-4b24-8d78-a03f5bd78408 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:48.237705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1357/2703] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:48 2022] PUT /identity/v3/projects/3e77bc90040c4171b10d8a6e0dda9a03/users/b3bc736526ba4bfca203384235ddb37e/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:48.241658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ff14a-98fe-44f4-86fb-5ce89b4ef2d3 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:48.251570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ff14a-98fe-44f4-86fb-5ce89b4ef2d3 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:48.306568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e04f1-9d92-4e4a-bb38-3f1e486cfc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:48.307184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e04f1-9d92-4e4a-bb38-3f1e486cfc20 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:48.307493 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e04f1-9d92-4e4a-bb38-3f1e486cfc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:48.307799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e04f1-9d92-4e4a-bb38-3f1e486cfc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:48.308518 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e04f1-9d92-4e4a-bb38-3f1e486cfc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:48.310084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e04f1-9d92-4e4a-bb38-3f1e486cfc20 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:48.319298 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1347/2704] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:48.323573 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d983a82-3399-4b24-8d78-a03f5bd78408 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:48.334516 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d983a82-3399-4b24-8d78-a03f5bd78408 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:48.339096 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b62a40-6009-451f-ae9a-ad7a8195a671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:48.339758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b62a40-6009-451f-ae9a-ad7a8195a671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:48.340038 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b62a40-6009-451f-ae9a-ad7a8195a671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:48.340308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b62a40-6009-451f-ae9a-ad7a8195a671 admin admin] PATH_INFO: `/v3/projects/3e77bc90040c4171b10d8a6e0dda9a03/users/b3bc736526ba4bfca203384235ddb37e/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:48.342189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b62a40-6009-451f-ae9a-ad7a8195a671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e77bc90040c4171b10d8a6e0dda9a03, user_id=b3bc736526ba4bfca203384235ddb37e, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:48.344841 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b62a40-6009-451f-ae9a-ad7a8195a671 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:48.355386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1358/2705] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:48 2022] PUT /identity/v3/projects/3e77bc90040c4171b10d8a6e0dda9a03/users/b3bc736526ba4bfca203384235ddb37e/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:48.366044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb424df-3f53-4c09-9612-c7783fece38e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:48.366329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb424df-3f53-4c09-9612-c7783fece38e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:48.366628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb424df-3f53-4c09-9612-c7783fece38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:48.394744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ebb424df-3f53-4c09-9612-c7783fece38e None None] MFA Rules not processed for user `b3bc736526ba4bfca203384235ddb37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:48.395956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ebb424df-3f53-4c09-9612-c7783fece38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:48.461650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1348/2706] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:48.472751 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b62a40-6009-451f-ae9a-ad7a8195a671 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:48.474431 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-49b62a40-6009-451f-ae9a-ad7a8195a671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:48.530206 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b62a40-6009-451f-ae9a-ad7a8195a671 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:48.534429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38b64-ff65-4bf4-8962-2273f01f41b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:48.535047 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38b64-ff65-4bf4-8962-2273f01f41b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:48.535326 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38b64-ff65-4bf4-8962-2273f01f41b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:48.535595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38b64-ff65-4bf4-8962-2273f01f41b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:48.544353 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa38b64-ff65-4bf4-8962-2273f01f41b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:48.545767 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa38b64-ff65-4bf4-8962-2273f01f41b0 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:48.555016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1359/2707] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:49.045379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb424df-3f53-4c09-9612-c7783fece38e None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:49.046844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ebb424df-3f53-4c09-9612-c7783fece38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:49.058439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb424df-3f53-4c09-9612-c7783fece38e None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:49.061858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8c048-391c-4506-b10d-3e05dd3a1004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:49.062440 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8c048-391c-4506-b10d-3e05dd3a1004 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:49.062709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8c048-391c-4506-b10d-3e05dd3a1004 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:49.063006 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8c048-391c-4506-b10d-3e05dd3a1004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:49.117902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8c048-391c-4506-b10d-3e05dd3a1004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:49.119327 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8c048-391c-4506-b10d-3e05dd3a1004 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:49.122686 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:49.130173 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1349/2708] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:53.117002 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38b64-ff65-4bf4-8962-2273f01f41b0 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:53.125417 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38b64-ff65-4bf4-8962-2273f01f41b0 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:53.130032 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997b704-1508-431c-b692-bdc5a4219798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:53.130619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997b704-1508-431c-b692-bdc5a4219798 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:53.130892 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997b704-1508-431c-b692-bdc5a4219798 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:53.131188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997b704-1508-431c-b692-bdc5a4219798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:53.151472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:53.187214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997b704-1508-431c-b692-bdc5a4219798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:53.188560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997b704-1508-431c-b692-bdc5a4219798 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:53.197942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1360/2709] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:55.227277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8c048-391c-4506-b10d-3e05dd3a1004 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:55.289926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8c048-391c-4506-b10d-3e05dd3a1004 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:55.294322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:55.294932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:55.295239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:55.295504 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] PATH_INFO: `/v3/users/d3c6de97ae5f4949afff96fd712245e6` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:55.296586 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c6de97ae5f4949afff96fd712245e6)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:55.299000 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:55.342924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c6de97ae5f4949afff96fd712245e6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.349463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c6de97ae5f4949afff96fd712245e6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.356140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c6de97ae5f4949afff96fd712245e6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.367911 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c6de97ae5f4949afff96fd712245e6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.368628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c6de97ae5f4949afff96fd712245e6'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.374200 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1350/2710] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:11:55 2022] DELETE /identity/v3/users/d3c6de97ae5f4949afff96fd712245e6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:55.379622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0d47-ce08-44a0-b6ea-f9ad4e8158d7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:55.380125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0d47-ce08-44a0-b6ea-f9ad4e8158d7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:55.380223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0d47-ce08-44a0-b6ea-f9ad4e8158d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:55.398507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7b6f0d47-ce08-44a0-b6ea-f9ad4e8158d7 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:55.399719 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7b6f0d47-ce08-44a0-b6ea-f9ad4e8158d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:55.459274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1361/2711] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:55.468836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:55.470308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:55.525910 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6867bbf-911a-4214-b3aa-752f5a1fb5a2 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:55.529541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8f290-9fe5-4b26-ab86-e3d3df8fee86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:55.530126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8f290-9fe5-4b26-ab86-e3d3df8fee86 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:55.530398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8f290-9fe5-4b26-ab86-e3d3df8fee86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:55.530662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8f290-9fe5-4b26-ab86-e3d3df8fee86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:55.539847 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8f290-9fe5-4b26-ab86-e3d3df8fee86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:55.541246 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8f290-9fe5-4b26-ab86-e3d3df8fee86 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:55.550912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1351/2712] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:55.812271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f0d47-ce08-44a0-b6ea-f9ad4e8158d7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:55.813734 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7b6f0d47-ce08-44a0-b6ea-f9ad4e8158d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:55.826349 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f0d47-ce08-44a0-b6ea-f9ad4e8158d7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:55.831022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:55.831649 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:55.831918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:55.832194 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] PATH_INFO: `/v3/projects/b79ef21804e94ba3b54458b2d7b2e21c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:55.833318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b79ef21804e94ba3b54458b2d7b2e21c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:55.835270 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:55.881370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b79ef21804e94ba3b54458b2d7b2e21c', 'request_id': 'req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.888697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b79ef21804e94ba3b54458b2d7b2e21c', 'request_id': 'req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.895230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b79ef21804e94ba3b54458b2d7b2e21c', 'request_id': 'req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.905333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] The token cache is being invalidate because project b79ef21804e94ba3b54458b2d7b2e21c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:11:55.905623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.906320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:55.907902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1362/2713] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:11:55 2022] DELETE /identity/v3/projects/b79ef21804e94ba3b54458b2d7b2e21c => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:55.912280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8f290-9fe5-4b26-ab86-e3d3df8fee86 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:55.921693 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8f290-9fe5-4b26-ab86-e3d3df8fee86 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:55.937724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:55.978615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:55.979267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:55.979543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:55.979810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] PATH_INFO: `/v3/users/6045b1bb79204607aeb7f9fb622be985` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:55.980902 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6045b1bb79204607aeb7f9fb622be985)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:55.982810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:56.026214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6045b1bb79204607aeb7f9fb622be985'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.032532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6045b1bb79204607aeb7f9fb622be985'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.039919 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6045b1bb79204607aeb7f9fb622be985'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.051280 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6045b1bb79204607aeb7f9fb622be985'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.051918 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6045b1bb79204607aeb7f9fb622be985'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.057562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1352/2714] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:11:55 2022] DELETE /identity/v3/users/6045b1bb79204607aeb7f9fb622be985 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:56.321745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:56.323188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:56.337276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e33aa2-c23e-4c5b-82f5-085916bd22a0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:56.381916 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:56.395529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:56.396162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:56.396439 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:56.396711 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] PATH_INFO: `/v3/projects/7d5f5b21b51d44a19db73e5cb52b7804` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:56.397837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5f5b21b51d44a19db73e5cb52b7804)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:56.399826 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:56.445429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5f5b21b51d44a19db73e5cb52b7804', 'request_id': 'req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.452236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5f5b21b51d44a19db73e5cb52b7804', 'request_id': 'req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.459017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5f5b21b51d44a19db73e5cb52b7804', 'request_id': 'req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.469311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] The token cache is being invalidate because project 7d5f5b21b51d44a19db73e5cb52b7804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:11:56.469595 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.470308 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.471823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1363/2715] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:11:56 2022] DELETE /identity/v3/projects/7d5f5b21b51d44a19db73e5cb52b7804 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:56.475750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:56.477169 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:56.489317 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850d986-4408-4551-a2b6-ee64785489fe admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:56.543463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:56.544084 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:56.544358 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:56.544627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] PATH_INFO: `/v3/users/b3bc736526ba4bfca203384235ddb37e` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:56.545738 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3bc736526ba4bfca203384235ddb37e)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:56.547672 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:56.588924 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:56.594665 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3bc736526ba4bfca203384235ddb37e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.602730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3bc736526ba4bfca203384235ddb37e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.610713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3bc736526ba4bfca203384235ddb37e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.622716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3bc736526ba4bfca203384235ddb37e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.623421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fbe0c30d-2f64-46e0-9767-c698c6095855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3bc736526ba4bfca203384235ddb37e'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:56.629916 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1353/2716] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:11:56 2022] DELETE /identity/v3/users/b3bc736526ba4bfca203384235ddb37e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:56.896071 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:56.897548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:56.910759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659e4fe-6fdf-445e-ad08-d9c9c0e72a7f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:56.970933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:56.971591 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:56.971868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:56.972138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] PATH_INFO: `/v3/projects/3e77bc90040c4171b10d8a6e0dda9a03` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:56.973242 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e77bc90040c4171b10d8a6e0dda9a03)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:56.975234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.020828 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e77bc90040c4171b10d8a6e0dda9a03', 'request_id': 'req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:57.027148 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e77bc90040c4171b10d8a6e0dda9a03', 'request_id': 'req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:57.033621 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e77bc90040c4171b10d8a6e0dda9a03', 'request_id': 'req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:57.043627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] The token cache is being invalidate because project 3e77bc90040c4171b10d8a6e0dda9a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:11:57.043913 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:57.044619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7e6f6f9-8483-4d2f-8d8c-0fd5eb0aa2c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:11:57.046149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1364/2717] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:11:56 2022] DELETE /identity/v3/projects/3e77bc90040c4171b10d8a6e0dda9a03 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:57.052629 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a56472-ac3f-4197-9349-70a0c0180796 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.052914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a56472-ac3f-4197-9349-70a0c0180796 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.053187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a56472-ac3f-4197-9349-70a0c0180796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.071935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-14a56472-ac3f-4197-9349-70a0c0180796 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:57.073119 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-14a56472-ac3f-4197-9349-70a0c0180796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:57.130720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1354/2718] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:57.139025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231afe43-0ce1-4a2b-bfef-f5fae0f3178e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.139314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231afe43-0ce1-4a2b-bfef-f5fae0f3178e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.139588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231afe43-0ce1-4a2b-bfef-f5fae0f3178e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.154247 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:57.159871 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-231afe43-0ce1-4a2b-bfef-f5fae0f3178e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:57.161437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-231afe43-0ce1-4a2b-bfef-f5fae0f3178e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:57.169600 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1365/2719] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:57.173740 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a56472-ac3f-4197-9349-70a0c0180796 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.183278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a56472-ac3f-4197-9349-70a0c0180796 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.187763 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cb05d-6420-468a-9c67-08067a65dfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.188344 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cb05d-6420-468a-9c67-08067a65dfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.188617 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cb05d-6420-468a-9c67-08067a65dfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.188882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cb05d-6420-468a-9c67-08067a65dfd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.190042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cb05d-6420-468a-9c67-08067a65dfd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:11:57.190337 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cb05d-6420-468a-9c67-08067a65dfd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.191938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cb05d-6420-468a-9c67-08067a65dfd1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.202995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1355/2720] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:11:57.207750 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231afe43-0ce1-4a2b-bfef-f5fae0f3178e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.216748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231afe43-0ce1-4a2b-bfef-f5fae0f3178e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.221364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66efe8-d9b4-454d-9617-79c11c0048c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.221940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66efe8-d9b4-454d-9617-79c11c0048c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.222217 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66efe8-d9b4-454d-9617-79c11c0048c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.222488 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66efe8-d9b4-454d-9617-79c11c0048c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.223143 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66efe8-d9b4-454d-9617-79c11c0048c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.225018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66efe8-d9b4-454d-9617-79c11c0048c1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.252410 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1366/2721] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:11:57 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:57.256320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cb05d-6420-468a-9c67-08067a65dfd1 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.266848 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cb05d-6420-468a-9c67-08067a65dfd1 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.325654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886491e-ccd5-4279-b50f-efb5fadd06b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.326256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886491e-ccd5-4279-b50f-efb5fadd06b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.326554 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886491e-ccd5-4279-b50f-efb5fadd06b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.326850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886491e-ccd5-4279-b50f-efb5fadd06b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.327508 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886491e-ccd5-4279-b50f-efb5fadd06b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.329466 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886491e-ccd5-4279-b50f-efb5fadd06b4 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.350482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1356/2722] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:11:57 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:57.354598 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66efe8-d9b4-454d-9617-79c11c0048c1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.365446 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66efe8-d9b4-454d-9617-79c11c0048c1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.370090 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54479303-e4df-40bf-a843-c93d662fd36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.370691 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54479303-e4df-40bf-a843-c93d662fd36a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.371004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54479303-e4df-40bf-a843-c93d662fd36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.371281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54479303-e4df-40bf-a843-c93d662fd36a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.371948 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54479303-e4df-40bf-a843-c93d662fd36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.373525 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54479303-e4df-40bf-a843-c93d662fd36a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.384485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1367/2723] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:57.388339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886491e-ccd5-4279-b50f-efb5fadd06b4 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.398100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886491e-ccd5-4279-b50f-efb5fadd06b4 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.402652 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b131c5-7b2f-4c2c-a560-5f83a8d2d8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.403277 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b131c5-7b2f-4c2c-a560-5f83a8d2d8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.403551 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b131c5-7b2f-4c2c-a560-5f83a8d2d8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.403821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b131c5-7b2f-4c2c-a560-5f83a8d2d8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.404458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b131c5-7b2f-4c2c-a560-5f83a8d2d8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.406066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b131c5-7b2f-4c2c-a560-5f83a8d2d8b2 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.416771 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1357/2724] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:57.421333 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54479303-e4df-40bf-a843-c93d662fd36a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.431838 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54479303-e4df-40bf-a843-c93d662fd36a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.436547 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0469c5-7cd6-42b3-a4c4-ae6d38b4e0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.437187 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0469c5-7cd6-42b3-a4c4-ae6d38b4e0b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.437498 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0469c5-7cd6-42b3-a4c4-ae6d38b4e0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.437765 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0469c5-7cd6-42b3-a4c4-ae6d38b4e0b0 admin admin] PATH_INFO: `/v3/projects/ff69d251e47a4423b0938a5eafc8b031/users/0bb6a8f428684d26a38de9453de971b4/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.452514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0469c5-7cd6-42b3-a4c4-ae6d38b4e0b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff69d251e47a4423b0938a5eafc8b031, user_id=0bb6a8f428684d26a38de9453de971b4, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.455204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0469c5-7cd6-42b3-a4c4-ae6d38b4e0b0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.467136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1368/2725] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:57 2022] PUT /identity/v3/projects/ff69d251e47a4423b0938a5eafc8b031/users/0bb6a8f428684d26a38de9453de971b4/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:11:57.534370 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f5da1-1144-4239-8a4e-de29e44408bd None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.534930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f5da1-1144-4239-8a4e-de29e44408bd None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.534930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f5da1-1144-4239-8a4e-de29e44408bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.561585 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c94f5da1-1144-4239-8a4e-de29e44408bd None None] MFA Rules not processed for user `0bb6a8f428684d26a38de9453de971b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:57.562767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c94f5da1-1144-4239-8a4e-de29e44408bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:57.573967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:57.626571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1358/2726] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:57.631145 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0469c5-7cd6-42b3-a4c4-ae6d38b4e0b0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.694080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0469c5-7cd6-42b3-a4c4-ae6d38b4e0b0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.698697 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb435a95-c1f5-4969-a461-188c9474801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.699327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb435a95-c1f5-4969-a461-188c9474801e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.703524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb435a95-c1f5-4969-a461-188c9474801e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.703524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb435a95-c1f5-4969-a461-188c9474801e admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.703524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb435a95-c1f5-4969-a461-188c9474801e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.703524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb435a95-c1f5-4969-a461-188c9474801e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.733501 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1369/2727] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:11:57 2022] POST /identity/v3/projects => generated 397 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:11:57.737605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f5da1-1144-4239-8a4e-de29e44408bd None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.748026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f5da1-1144-4239-8a4e-de29e44408bd None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.803035 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd954f44-362e-4e9f-9b1c-1c399c8d79ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.803640 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd954f44-362e-4e9f-9b1c-1c399c8d79ab admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.803907 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd954f44-362e-4e9f-9b1c-1c399c8d79ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.804214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd954f44-362e-4e9f-9b1c-1c399c8d79ab admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.804821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd954f44-362e-4e9f-9b1c-1c399c8d79ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.806706 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd954f44-362e-4e9f-9b1c-1c399c8d79ab admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.827593 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1359/2728] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:11:57 2022] POST /identity/v3/users => generated 305 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:11:57.832301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb435a95-c1f5-4969-a461-188c9474801e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.842281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb435a95-c1f5-4969-a461-188c9474801e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.846815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3eb1e-4bc6-4557-8448-d15abc43b282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.847449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3eb1e-4bc6-4557-8448-d15abc43b282 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.847724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3eb1e-4bc6-4557-8448-d15abc43b282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.847994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3eb1e-4bc6-4557-8448-d15abc43b282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.848622 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3eb1e-4bc6-4557-8448-d15abc43b282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.850228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3eb1e-4bc6-4557-8448-d15abc43b282 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.860444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1370/2729] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:57.864273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd954f44-362e-4e9f-9b1c-1c399c8d79ab admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.873866 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd954f44-362e-4e9f-9b1c-1c399c8d79ab admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.879250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43822541-0d66-4978-96c5-5c9a3e420c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.879880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43822541-0d66-4978-96c5-5c9a3e420c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.880158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43822541-0d66-4978-96c5-5c9a3e420c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.880429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43822541-0d66-4978-96c5-5c9a3e420c65 admin admin] PATH_INFO: `/v3/projects/d4e50d7e928246859e83e6f015ae4f8a/users/36439698687244ad9c7825d4b19420e9/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.895183 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43822541-0d66-4978-96c5-5c9a3e420c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e50d7e928246859e83e6f015ae4f8a, user_id=36439698687244ad9c7825d4b19420e9, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.897783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43822541-0d66-4978-96c5-5c9a3e420c65 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.910088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1360/2730] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:57 2022] PUT /identity/v3/projects/d4e50d7e928246859e83e6f015ae4f8a/users/36439698687244ad9c7825d4b19420e9/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:11:57.913929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3eb1e-4bc6-4557-8448-d15abc43b282 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:57.923524 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3eb1e-4bc6-4557-8448-d15abc43b282 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:57.979444 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc813d-73ed-42ec-bba4-8cd866231665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:57.980062 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc813d-73ed-42ec-bba4-8cd866231665 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:57.980336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc813d-73ed-42ec-bba4-8cd866231665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:57.980604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc813d-73ed-42ec-bba4-8cd866231665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:57.981229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc813d-73ed-42ec-bba4-8cd866231665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:57.982800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc813d-73ed-42ec-bba4-8cd866231665 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:57.992893 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1371/2731] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:11:57.997089 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43822541-0d66-4978-96c5-5c9a3e420c65 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:58.007544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43822541-0d66-4978-96c5-5c9a3e420c65 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:58.012272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35ad2-8093-4315-86aa-8e7b7ab7c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:58.012944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35ad2-8093-4315-86aa-8e7b7ab7c5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:58.013222 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35ad2-8093-4315-86aa-8e7b7ab7c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:58.013492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35ad2-8093-4315-86aa-8e7b7ab7c5bd admin admin] PATH_INFO: `/v3/projects/d4e50d7e928246859e83e6f015ae4f8a/users/36439698687244ad9c7825d4b19420e9/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:58.015446 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35ad2-8093-4315-86aa-8e7b7ab7c5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e50d7e928246859e83e6f015ae4f8a, user_id=36439698687244ad9c7825d4b19420e9, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:58.018598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35ad2-8093-4315-86aa-8e7b7ab7c5bd admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:58.029336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1361/2732] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:11:57 2022] PUT /identity/v3/projects/d4e50d7e928246859e83e6f015ae4f8a/users/36439698687244ad9c7825d4b19420e9/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:11:58.040140 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dd4b4-cf81-40ce-aa6c-4d968b1733d4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:58.040427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dd4b4-cf81-40ce-aa6c-4d968b1733d4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:58.040702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dd4b4-cf81-40ce-aa6c-4d968b1733d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:58.065855 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:11:58.068692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9f4dd4b4-cf81-40ce-aa6c-4d968b1733d4 None None] MFA Rules not processed for user `36439698687244ad9c7825d4b19420e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:11:58.070692 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f4dd4b4-cf81-40ce-aa6c-4d968b1733d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:58.138644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1372/2733] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:11:58 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:11:58.150069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35ad2-8093-4315-86aa-8e7b7ab7c5bd admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:58.151777 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-cac35ad2-8093-4315-86aa-8e7b7ab7c5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:11:58.207355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35ad2-8093-4315-86aa-8e7b7ab7c5bd admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:58.210977 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2feca2-17d2-4f72-93c2-c8f0bec9e4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:58.211564 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2feca2-17d2-4f72-93c2-c8f0bec9e4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:58.211840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2feca2-17d2-4f72-93c2-c8f0bec9e4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:58.212110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2feca2-17d2-4f72-93c2-c8f0bec9e4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:58.221321 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2feca2-17d2-4f72-93c2-c8f0bec9e4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:58.222720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2feca2-17d2-4f72-93c2-c8f0bec9e4e8 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:58.231788 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1362/2734] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:58 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:11:58.305996 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dd4b4-cf81-40ce-aa6c-4d968b1733d4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:11:58.315624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dd4b4-cf81-40ce-aa6c-4d968b1733d4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:11:58.319277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8ddfb-9426-49d8-974d-a8d3cb2be322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:11:58.319811 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8ddfb-9426-49d8-974d-a8d3cb2be322 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:11:58.320087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8ddfb-9426-49d8-974d-a8d3cb2be322 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:11:58.320356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8ddfb-9426-49d8-974d-a8d3cb2be322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:11:58.376383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8ddfb-9426-49d8-974d-a8d3cb2be322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:11:58.377896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8ddfb-9426-49d8-974d-a8d3cb2be322 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:11:58.387179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1373/2735] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:11:58 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:12:02.403742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2feca2-17d2-4f72-93c2-c8f0bec9e4e8 service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:02.413620 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:02.464794 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2feca2-17d2-4f72-93c2-c8f0bec9e4e8 service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:02.469336 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:02.469949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:02.470732 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:02.471042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] PATH_INFO: `/v3/users/0bb6a8f428684d26a38de9453de971b4` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:02.472141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb6a8f428684d26a38de9453de971b4)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:02.474074 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:02.519406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb6a8f428684d26a38de9453de971b4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:02.526233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb6a8f428684d26a38de9453de971b4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:02.533741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb6a8f428684d26a38de9453de971b4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:02.545146 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb6a8f428684d26a38de9453de971b4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:02.545774 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb6a8f428684d26a38de9453de971b4'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:02.551450 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1363/2736] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:12:02 2022] DELETE /identity/v3/users/0bb6a8f428684d26a38de9453de971b4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:02.555870 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ed66b-71ff-43a4-8421-0d27d61999b8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:02.556377 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ed66b-71ff-43a4-8421-0d27d61999b8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:02.556480 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ed66b-71ff-43a4-8421-0d27d61999b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:02.574554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5f6ed66b-71ff-43a4-8421-0d27d61999b8 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:02.575776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5f6ed66b-71ff-43a4-8421-0d27d61999b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:02.633731 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1374/2737] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:02.643974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:02.645363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:02.702490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a5362-d144-44b7-a4f3-0b1af3b5abd6 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:02.706573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554ffc3-4c67-4dbd-bb6d-6f4b1f863d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:02.707197 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554ffc3-4c67-4dbd-bb6d-6f4b1f863d5f service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:02.707470 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554ffc3-4c67-4dbd-bb6d-6f4b1f863d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:02.707737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554ffc3-4c67-4dbd-bb6d-6f4b1f863d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:02.717421 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554ffc3-4c67-4dbd-bb6d-6f4b1f863d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:02.718885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554ffc3-4c67-4dbd-bb6d-6f4b1f863d5f service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:02.728654 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1364/2738] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:12:02.994025 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ed66b-71ff-43a4-8421-0d27d61999b8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:02.995613 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5f6ed66b-71ff-43a4-8421-0d27d61999b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:03.008604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ed66b-71ff-43a4-8421-0d27d61999b8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:03.013320 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:03.013898 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:03.014182 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:03.014457 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] PATH_INFO: `/v3/projects/ff69d251e47a4423b0938a5eafc8b031` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:03.015987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff69d251e47a4423b0938a5eafc8b031)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:03.018543 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:03.054059 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:03.067228 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff69d251e47a4423b0938a5eafc8b031', 'request_id': 'req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.074409 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff69d251e47a4423b0938a5eafc8b031', 'request_id': 'req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.081803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff69d251e47a4423b0938a5eafc8b031', 'request_id': 'req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.092372 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] The token cache is being invalidate because project ff69d251e47a4423b0938a5eafc8b031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:12:03.092657 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.093297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.094813 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1375/2739] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:12:02 2022] DELETE /identity/v3/projects/ff69d251e47a4423b0938a5eafc8b031 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:03.099256 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554ffc3-4c67-4dbd-bb6d-6f4b1f863d5f service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:03.108731 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554ffc3-4c67-4dbd-bb6d-6f4b1f863d5f service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:03.136985 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:03.166069 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:03.166683 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:03.166974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:03.167266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] PATH_INFO: `/v3/users/36439698687244ad9c7825d4b19420e9` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:03.168366 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] RBAC: Authorizing `identity:delete_user(user_id=36439698687244ad9c7825d4b19420e9)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:03.170270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:03.215389 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36439698687244ad9c7825d4b19420e9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.223205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36439698687244ad9c7825d4b19420e9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.230002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36439698687244ad9c7825d4b19420e9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.242178 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36439698687244ad9c7825d4b19420e9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.242807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0b530647-2c83-42fd-957f-3d192db3c99c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36439698687244ad9c7825d4b19420e9'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.248635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1365/2740] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:12:03 2022] DELETE /identity/v3/users/36439698687244ad9c7825d4b19420e9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:03.537266 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:03.538842 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:03.556335 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7921b5-ee1a-4ba7-85fb-3645ee8d847a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:03.612302 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:03.612854 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:03.613131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:03.613419 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] PATH_INFO: `/v3/projects/d4e50d7e928246859e83e6f015ae4f8a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:03.614603 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e50d7e928246859e83e6f015ae4f8a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:03.616536 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:03.661164 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e50d7e928246859e83e6f015ae4f8a', 'request_id': 'req-ae3821d4-af10-4409-b2d0-cc57665679ba'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.667705 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e50d7e928246859e83e6f015ae4f8a', 'request_id': 'req-ae3821d4-af10-4409-b2d0-cc57665679ba'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.673839 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e50d7e928246859e83e6f015ae4f8a', 'request_id': 'req-ae3821d4-af10-4409-b2d0-cc57665679ba'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.683890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] The token cache is being invalidate because project d4e50d7e928246859e83e6f015ae4f8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:12:03.684177 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.684798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ae3821d4-af10-4409-b2d0-cc57665679ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:03.686251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1376/2741] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:12:03 2022] DELETE /identity/v3/projects/d4e50d7e928246859e83e6f015ae4f8a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:03.692677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4620a-53dd-40f5-918f-ca222aa33d8d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:03.693159 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4620a-53dd-40f5-918f-ca222aa33d8d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:03.693278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4620a-53dd-40f5-918f-ca222aa33d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:03.712536 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-49d4620a-53dd-40f5-918f-ca222aa33d8d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:03.713704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-49d4620a-53dd-40f5-918f-ca222aa33d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:03.770867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1366/2742] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:03.778663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfccaa-2f12-400e-94ca-be3f8e51e81e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:03.778965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfccaa-2f12-400e-94ca-be3f8e51e81e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:03.779274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfccaa-2f12-400e-94ca-be3f8e51e81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:03.798191 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0cbfccaa-2f12-400e-94ca-be3f8e51e81e None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:03.799414 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0cbfccaa-2f12-400e-94ca-be3f8e51e81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:03.807172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1377/2743] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:03.811199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4620a-53dd-40f5-918f-ca222aa33d8d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:03.821093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4620a-53dd-40f5-918f-ca222aa33d8d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:03.825690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65477808-b73f-47bc-bdf9-2df84cdcaf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:03.826281 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65477808-b73f-47bc-bdf9-2df84cdcaf79 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:03.826572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65477808-b73f-47bc-bdf9-2df84cdcaf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:03.826832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65477808-b73f-47bc-bdf9-2df84cdcaf79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:03.827519 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65477808-b73f-47bc-bdf9-2df84cdcaf79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:12:03.827838 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65477808-b73f-47bc-bdf9-2df84cdcaf79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:03.829467 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65477808-b73f-47bc-bdf9-2df84cdcaf79 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:03.841420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1367/2744] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:12:03.845554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfccaa-2f12-400e-94ca-be3f8e51e81e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:03.854540 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfccaa-2f12-400e-94ca-be3f8e51e81e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:03.859213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d5215-5bbb-40ee-a70e-cb2a8b91afc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:03.859802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d5215-5bbb-40ee-a70e-cb2a8b91afc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:03.860081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d5215-5bbb-40ee-a70e-cb2a8b91afc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:03.860350 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d5215-5bbb-40ee-a70e-cb2a8b91afc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:03.860976 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d5215-5bbb-40ee-a70e-cb2a8b91afc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:03.862897 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d5215-5bbb-40ee-a70e-cb2a8b91afc1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:03.889348 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1378/2745] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:12:03 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:12:03.893950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65477808-b73f-47bc-bdf9-2df84cdcaf79 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:03.903649 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65477808-b73f-47bc-bdf9-2df84cdcaf79 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:03.935232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:03.960392 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb810b-7559-41f3-8e8e-7651f122bdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:03.960992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb810b-7559-41f3-8e8e-7651f122bdba admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:03.961266 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb810b-7559-41f3-8e8e-7651f122bdba admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:03.961564 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb810b-7559-41f3-8e8e-7651f122bdba admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:03.962180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb810b-7559-41f3-8e8e-7651f122bdba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:03.964167 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb810b-7559-41f3-8e8e-7651f122bdba admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:03.984728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1368/2746] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:12:03 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:12:03.989312 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d5215-5bbb-40ee-a70e-cb2a8b91afc1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:03.999519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d5215-5bbb-40ee-a70e-cb2a8b91afc1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:04.004385 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605240e9-86e0-495e-b6d6-724b11147feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:04.004997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605240e9-86e0-495e-b6d6-724b11147feb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:04.005268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605240e9-86e0-495e-b6d6-724b11147feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:04.005535 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605240e9-86e0-495e-b6d6-724b11147feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:04.006190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605240e9-86e0-495e-b6d6-724b11147feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:04.007803 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605240e9-86e0-495e-b6d6-724b11147feb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:04.018004 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1379/2747] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:12:04.021814 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb810b-7559-41f3-8e8e-7651f122bdba admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:04.031708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb810b-7559-41f3-8e8e-7651f122bdba admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:04.036821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3087fe-362e-4e18-8299-92f2f7b67c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:04.037399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3087fe-362e-4e18-8299-92f2f7b67c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:04.037673 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3087fe-362e-4e18-8299-92f2f7b67c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:04.037939 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3087fe-362e-4e18-8299-92f2f7b67c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:04.038562 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3087fe-362e-4e18-8299-92f2f7b67c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:04.040161 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3087fe-362e-4e18-8299-92f2f7b67c2b admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:04.050207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1369/2748] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:12:04.054022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605240e9-86e0-495e-b6d6-724b11147feb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:04.063529 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605240e9-86e0-495e-b6d6-724b11147feb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:04.068042 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3f5e0-4131-4a32-94a2-838b55fa709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:04.068670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3f5e0-4131-4a32-94a2-838b55fa709f admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:04.068945 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3f5e0-4131-4a32-94a2-838b55fa709f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:04.069214 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3f5e0-4131-4a32-94a2-838b55fa709f admin admin] PATH_INFO: `/v3/projects/1fc4b8f7573d4c13a122c9a32ed71403/users/e13b0d8cddd3413c8af575c061671515/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:04.077426 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:04.084890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3f5e0-4131-4a32-94a2-838b55fa709f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc4b8f7573d4c13a122c9a32ed71403, user_id=e13b0d8cddd3413c8af575c061671515, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:04.088131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3f5e0-4131-4a32-94a2-838b55fa709f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:04.101134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1380/2749] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:12:04 2022] PUT /identity/v3/projects/1fc4b8f7573d4c13a122c9a32ed71403/users/e13b0d8cddd3413c8af575c061671515/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:04.112449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02409255-2f69-4167-93ca-bdc20558442e None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:04.112727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02409255-2f69-4167-93ca-bdc20558442e None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:04.113048 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02409255-2f69-4167-93ca-bdc20558442e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:04.139618 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-02409255-2f69-4167-93ca-bdc20558442e None None] MFA Rules not processed for user `e13b0d8cddd3413c8af575c061671515`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:04.140801 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-02409255-2f69-4167-93ca-bdc20558442e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:04.202571 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1370/2750] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:04.213503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3f5e0-4131-4a32-94a2-838b55fa709f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:04.215230 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d7f3f5e0-4131-4a32-94a2-838b55fa709f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:04.271075 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3f5e0-4131-4a32-94a2-838b55fa709f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:04.275987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fc585-06d9-4be3-8da4-af731c1635bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:04.276592 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fc585-06d9-4be3-8da4-af731c1635bb service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:04.276868 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fc585-06d9-4be3-8da4-af731c1635bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:04.277138 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fc585-06d9-4be3-8da4-af731c1635bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:04.285280 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fc585-06d9-4be3-8da4-af731c1635bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:04.286684 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fc585-06d9-4be3-8da4-af731c1635bb service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:04.295201 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1381/2751] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:12:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:12:05.645650 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19995000-1528-4c37-a506-5b9827a94277 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:05.646147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19995000-1528-4c37-a506-5b9827a94277 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:05.646262 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19995000-1528-4c37-a506-5b9827a94277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:05.665603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-19995000-1528-4c37-a506-5b9827a94277 None None] MFA Rules not processed for user `e13b0d8cddd3413c8af575c061671515`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:05.666702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-19995000-1528-4c37-a506-5b9827a94277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:05.673449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1371/2752] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:14.291389 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:15.660383 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:21.957616 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75301a9e-f33c-4f81-98e1-4b2a14a71c26 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:21.958121 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75301a9e-f33c-4f81-98e1-4b2a14a71c26 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:21.958519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75301a9e-f33c-4f81-98e1-4b2a14a71c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:21.990472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-75301a9e-f33c-4f81-98e1-4b2a14a71c26 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:21.991997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-75301a9e-f33c-4f81-98e1-4b2a14a71c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:22.058769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1382/2753] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:12:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:22.106636 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19995000-1528-4c37-a506-5b9827a94277 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:22.108153 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-19995000-1528-4c37-a506-5b9827a94277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:22.182264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19995000-1528-4c37-a506-5b9827a94277 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:22.187233 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5fafd-002d-4280-9eca-fdf2c3b326b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:22.187854 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5fafd-002d-4280-9eca-fdf2c3b326b0 service swift] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:22.188128 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5fafd-002d-4280-9eca-fdf2c3b326b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:22.189045 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5fafd-002d-4280-9eca-fdf2c3b326b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:22.197833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5fafd-002d-4280-9eca-fdf2c3b326b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:22.199535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5fafd-002d-4280-9eca-fdf2c3b326b0 service swift] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:22.207890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1372/2754] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:12:22.723055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112064-cdb0-4913-9ea2-ceaae1fcc9e2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:22.723342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112064-cdb0-4913-9ea2-ceaae1fcc9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:22.723617 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112064-cdb0-4913-9ea2-ceaae1fcc9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:22.742403 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3d112064-cdb0-4913-9ea2-ceaae1fcc9e2 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:22.743608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3d112064-cdb0-4913-9ea2-ceaae1fcc9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:22.751368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1383/2755] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 10 10:12:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:22.763361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5fafd-002d-4280-9eca-fdf2c3b326b0 service swift] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:22.774279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5fafd-002d-4280-9eca-fdf2c3b326b0 service swift] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:22.779322 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19bf40-77b8-4944-a626-a8c82c1ed31b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:22.779908 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19bf40-77b8-4944-a626-a8c82c1ed31b service swift] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:22.780189 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19bf40-77b8-4944-a626-a8c82c1ed31b service swift] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:22.780458 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19bf40-77b8-4944-a626-a8c82c1ed31b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:22.788339 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19bf40-77b8-4944-a626-a8c82c1ed31b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:22.789882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19bf40-77b8-4944-a626-a8c82c1ed31b service swift] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:22.798206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1373/2756] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:12:23.008918 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc9849-aaec-4cd2-9632-ded64347569d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:23.009365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc9849-aaec-4cd2-9632-ded64347569d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:23.009586 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc9849-aaec-4cd2-9632-ded64347569d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:23.031057 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c5dc9849-aaec-4cd2-9632-ded64347569d None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:23.032313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c5dc9849-aaec-4cd2-9632-ded64347569d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:23.039961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1384/2757] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 10 10:12:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:12:23.052083 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19bf40-77b8-4944-a626-a8c82c1ed31b service swift] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:23.062401 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19bf40-77b8-4944-a626-a8c82c1ed31b service swift] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:23.068052 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9be7b7-2226-451c-b95b-2e9796320bcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:23.068744 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be7b7-2226-451c-b95b-2e9796320bcb service swift] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:23.069027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be7b7-2226-451c-b95b-2e9796320bcb service swift] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:23.069298 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be7b7-2226-451c-b95b-2e9796320bcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:23.077782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9be7b7-2226-451c-b95b-2e9796320bcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:23.079427 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9be7b7-2226-451c-b95b-2e9796320bcb service swift] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:23.088589 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1374/2758] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:12:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:12:23.425229 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86635a81-53da-40f4-adc2-02fe37bffaa8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:23.425404 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86635a81-53da-40f4-adc2-02fe37bffaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:23.425678 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86635a81-53da-40f4-adc2-02fe37bffaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:23.452776 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-86635a81-53da-40f4-adc2-02fe37bffaa8 None None] MFA Rules not processed for user `36f9577e047a406480fadc4b9f8c6234`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:23.454721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-86635a81-53da-40f4-adc2-02fe37bffaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:23.472364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1385/2759] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 10 10:12:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:23.486896 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9be7b7-2226-451c-b95b-2e9796320bcb service swift] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:23.498250 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9be7b7-2226-451c-b95b-2e9796320bcb service swift] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:23.504182 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512c77a-e0c1-4d6c-9ce3-ee15fd80eaef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af29f3451f654f53ac98239a2b28d128', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:23.504772 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512c77a-e0c1-4d6c-9ce3-ee15fd80eaef service swift] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:23.505046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512c77a-e0c1-4d6c-9ce3-ee15fd80eaef service swift] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:23.505308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512c77a-e0c1-4d6c-9ce3-ee15fd80eaef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:23.513580 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512c77a-e0c1-4d6c-9ce3-ee15fd80eaef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:23.515158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512c77a-e0c1-4d6c-9ce3-ee15fd80eaef service swift] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:23.523899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1375/2760] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 10 10:12:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:12:25.190131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88ef23-f568-4073-af9e-ddfba81bb355 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:25.190648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88ef23-f568-4073-af9e-ddfba81bb355 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:25.190748 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88ef23-f568-4073-af9e-ddfba81bb355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:25.210079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ab88ef23-f568-4073-af9e-ddfba81bb355 None None] MFA Rules not processed for user `67a34ddb337649ee948d319b0bf5fab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:25.211305 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ab88ef23-f568-4073-af9e-ddfba81bb355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:25.264968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1386/2761] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:25.676288 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512c77a-e0c1-4d6c-9ce3-ee15fd80eaef service swift] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:25.739209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512c77a-e0c1-4d6c-9ce3-ee15fd80eaef service swift] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:25.743812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:25.744432 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:25.744707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:25.744978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] PATH_INFO: `/v3/users/e13b0d8cddd3413c8af575c061671515` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:25.746086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13b0d8cddd3413c8af575c061671515)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:25.748025 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:25.794928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13b0d8cddd3413c8af575c061671515'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:25.798439 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:25.803405 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13b0d8cddd3413c8af575c061671515'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:25.812605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13b0d8cddd3413c8af575c061671515'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:25.826680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13b0d8cddd3413c8af575c061671515'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:25.827377 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13b0d8cddd3413c8af575c061671515'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:25.833760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1376/2762] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:12:25 2022] DELETE /identity/v3/users/e13b0d8cddd3413c8af575c061671515 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:25.839279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c17e2-d1de-4521-b347-188629dd67d1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:25.839820 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c17e2-d1de-4521-b347-188629dd67d1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:25.840538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c17e2-d1de-4521-b347-188629dd67d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:25.866167 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-119c17e2-d1de-4521-b347-188629dd67d1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:25.867405 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-119c17e2-d1de-4521-b347-188629dd67d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:25.937969 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1387/2763] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:25.948192 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:25.949746 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:26.008465 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1910e-3a22-48ef-8c31-6f8b9d304396 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:26.012082 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f935cf-2b72-4064-818f-6b69436847b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:26.012682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f935cf-2b72-4064-818f-6b69436847b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.012960 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f935cf-2b72-4064-818f-6b69436847b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.013231 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f935cf-2b72-4064-818f-6b69436847b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.022254 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f935cf-2b72-4064-818f-6b69436847b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:26.023689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f935cf-2b72-4064-818f-6b69436847b1 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:26.033351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1377/2764] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:12:26.217639 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c17e2-d1de-4521-b347-188629dd67d1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:26.220174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-119c17e2-d1de-4521-b347-188629dd67d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:26.233464 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c17e2-d1de-4521-b347-188629dd67d1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:26.238216 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:26.239115 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.239395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.239792 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] PATH_INFO: `/v3/projects/1fc4b8f7573d4c13a122c9a32ed71403` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.240994 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc4b8f7573d4c13a122c9a32ed71403)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:26.243091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:26.255756 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:26.296223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc4b8f7573d4c13a122c9a32ed71403', 'request_id': 'req-90e692fe-56a1-4ce1-8828-11ad9980637f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:26.304125 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc4b8f7573d4c13a122c9a32ed71403', 'request_id': 'req-90e692fe-56a1-4ce1-8828-11ad9980637f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:26.310484 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc4b8f7573d4c13a122c9a32ed71403', 'request_id': 'req-90e692fe-56a1-4ce1-8828-11ad9980637f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:26.321551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] The token cache is being invalidate because project 1fc4b8f7573d4c13a122c9a32ed71403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:12:26.321846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:26.322519 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-90e692fe-56a1-4ce1-8828-11ad9980637f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:26.324070 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1388/2765] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:12:26 2022] DELETE /identity/v3/projects/1fc4b8f7573d4c13a122c9a32ed71403 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:26.329177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea5ccf-b795-4975-8291-82b46085c38a None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.329525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea5ccf-b795-4975-8291-82b46085c38a None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.329804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea5ccf-b795-4975-8291-82b46085c38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.352512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-bbea5ccf-b795-4975-8291-82b46085c38a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:26.353775 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-bbea5ccf-b795-4975-8291-82b46085c38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:26.422247 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1378/2766] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:26.430561 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4de231-ed5b-4f38-bc0f-561e3184a3b9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.430851 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4de231-ed5b-4f38-bc0f-561e3184a3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.431162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4de231-ed5b-4f38-bc0f-561e3184a3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.451188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2f4de231-ed5b-4f38-bc0f-561e3184a3b9 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:26.452400 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2f4de231-ed5b-4f38-bc0f-561e3184a3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:26.460988 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1389/2767] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:12:26.465453 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea5ccf-b795-4975-8291-82b46085c38a None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:26.476526 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea5ccf-b795-4975-8291-82b46085c38a None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:26.481092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba562847-9925-45be-b903-d155d5bed836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:26.481679 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562847-9925-45be-b903-d155d5bed836 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.481950 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562847-9925-45be-b903-d155d5bed836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.482214 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562847-9925-45be-b903-d155d5bed836 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.482873 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba562847-9925-45be-b903-d155d5bed836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:12:26.483207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba562847-9925-45be-b903-d155d5bed836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:26.484767 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba562847-9925-45be-b903-d155d5bed836 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:26.496265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1379/2768] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:12:26.501321 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4de231-ed5b-4f38-bc0f-561e3184a3b9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:26.510517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4de231-ed5b-4f38-bc0f-561e3184a3b9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:26.515249 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ba937-fe98-44b1-a862-696f70747fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:26.515829 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ba937-fe98-44b1-a862-696f70747fba admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.516109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ba937-fe98-44b1-a862-696f70747fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.517043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ba937-fe98-44b1-a862-696f70747fba admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.517721 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ba937-fe98-44b1-a862-696f70747fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:26.519641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ba937-fe98-44b1-a862-696f70747fba admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:26.546192 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1390/2769] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:12:26 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:12:26.550758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba562847-9925-45be-b903-d155d5bed836 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:26.560480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba562847-9925-45be-b903-d155d5bed836 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:26.617130 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c4948-a1c9-4b5d-abe9-dd80f372839f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:26.617727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c4948-a1c9-4b5d-abe9-dd80f372839f admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.618004 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c4948-a1c9-4b5d-abe9-dd80f372839f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.618273 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c4948-a1c9-4b5d-abe9-dd80f372839f admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.618875 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c4948-a1c9-4b5d-abe9-dd80f372839f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:26.620870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c4948-a1c9-4b5d-abe9-dd80f372839f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:26.629447 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:26.642742 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1380/2770] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:12:26 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:12:26.646981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ba937-fe98-44b1-a862-696f70747fba admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:26.657162 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ba937-fe98-44b1-a862-696f70747fba admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:26.661968 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82e9ed-f45e-46b3-83a3-753b7b05c524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:26.662604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82e9ed-f45e-46b3-83a3-753b7b05c524 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.662880 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82e9ed-f45e-46b3-83a3-753b7b05c524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.663196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82e9ed-f45e-46b3-83a3-753b7b05c524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.663867 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82e9ed-f45e-46b3-83a3-753b7b05c524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:26.665495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82e9ed-f45e-46b3-83a3-753b7b05c524 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:26.675888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1391/2771] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:12:26.679707 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c4948-a1c9-4b5d-abe9-dd80f372839f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:26.689960 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c4948-a1c9-4b5d-abe9-dd80f372839f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:26.694570 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bee77b-8191-488a-bf15-bcb04da502b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:26.695186 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee77b-8191-488a-bf15-bcb04da502b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.695461 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee77b-8191-488a-bf15-bcb04da502b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.695734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee77b-8191-488a-bf15-bcb04da502b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.696369 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bee77b-8191-488a-bf15-bcb04da502b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:26.697936 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bee77b-8191-488a-bf15-bcb04da502b0 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:26.709046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1381/2772] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:12:26.713226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82e9ed-f45e-46b3-83a3-753b7b05c524 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:26.724663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82e9ed-f45e-46b3-83a3-753b7b05c524 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:26.729390 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015a5fd-9ea2-48e2-84bf-9129984dc619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:26.730066 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015a5fd-9ea2-48e2-84bf-9129984dc619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.730332 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015a5fd-9ea2-48e2-84bf-9129984dc619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.730604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015a5fd-9ea2-48e2-84bf-9129984dc619 admin admin] PATH_INFO: `/v3/projects/7e14a0b0b9524e61a6e90e2cbfcdb9dc/users/4dafffea269649788eea1325d562159c/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.746139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015a5fd-9ea2-48e2-84bf-9129984dc619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e14a0b0b9524e61a6e90e2cbfcdb9dc, user_id=4dafffea269649788eea1325d562159c, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:26.748853 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015a5fd-9ea2-48e2-84bf-9129984dc619 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:26.762026 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1392/2773] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:12:26 2022] PUT /identity/v3/projects/7e14a0b0b9524e61a6e90e2cbfcdb9dc/users/4dafffea269649788eea1325d562159c/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:26.773379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea06708-3081-4489-a2e4-39f740377608 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.773677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea06708-3081-4489-a2e4-39f740377608 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.773956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea06708-3081-4489-a2e4-39f740377608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.804276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-3ea06708-3081-4489-a2e4-39f740377608 None None] MFA Rules not processed for user `4dafffea269649788eea1325d562159c`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:26.806026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3ea06708-3081-4489-a2e4-39f740377608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:26.882737 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1382/2774] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:12:26.894655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015a5fd-9ea2-48e2-84bf-9129984dc619 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:26.897268 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8015a5fd-9ea2-48e2-84bf-9129984dc619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:26.969105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015a5fd-9ea2-48e2-84bf-9129984dc619 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:26.973133 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e8915-3267-435e-b2a1-113a56e3d6ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:26.973740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e8915-3267-435e-b2a1-113a56e3d6ff service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:26.974016 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e8915-3267-435e-b2a1-113a56e3d6ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:26.974277 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e8915-3267-435e-b2a1-113a56e3d6ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:26.983149 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e8915-3267-435e-b2a1-113a56e3d6ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:26.984670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e8915-3267-435e-b2a1-113a56e3d6ff service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:26.994207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1393/2775] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:12:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:12:36.879674 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:36.990418 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:45.269819 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea06708-3081-4489-a2e4-39f740377608 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:45.331797 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea06708-3081-4489-a2e4-39f740377608 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:45.336228 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:45.336840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:45.337116 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:45.337388 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] PATH_INFO: `/v3/users/4dafffea269649788eea1325d562159c` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:45.338481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dafffea269649788eea1325d562159c)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:45.340409 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:45.384664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dafffea269649788eea1325d562159c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.391143 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dafffea269649788eea1325d562159c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.398131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dafffea269649788eea1325d562159c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.410081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dafffea269649788eea1325d562159c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.410747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dafffea269649788eea1325d562159c'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.416541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1383/2776] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:12:45 2022] DELETE /identity/v3/users/4dafffea269649788eea1325d562159c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:45.420946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1917e4-f7af-4144-b1b5-b33ea2518788 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:45.421368 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1917e4-f7af-4144-b1b5-b33ea2518788 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:45.421480 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1917e4-f7af-4144-b1b5-b33ea2518788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:45.440926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8d1917e4-f7af-4144-b1b5-b33ea2518788 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:12:45.442110 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8d1917e4-f7af-4144-b1b5-b33ea2518788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:45.498116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1394/2777] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:12:45.508512 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:45.509882 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:45.566525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85a66f-a052-4657-a52a-644dd17d885a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:45.570160 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63aadbf-e9bd-45d9-b82b-31206f4f6063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:45.570747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63aadbf-e9bd-45d9-b82b-31206f4f6063 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:45.571050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63aadbf-e9bd-45d9-b82b-31206f4f6063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:45.571320 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63aadbf-e9bd-45d9-b82b-31206f4f6063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:45.580076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63aadbf-e9bd-45d9-b82b-31206f4f6063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:45.581496 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63aadbf-e9bd-45d9-b82b-31206f4f6063 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:45.591311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1384/2778] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:12:45.852219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1917e4-f7af-4144-b1b5-b33ea2518788 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:12:45.853663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8d1917e4-f7af-4144-b1b5-b33ea2518788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:12:45.868022 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1917e4-f7af-4144-b1b5-b33ea2518788 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:12:45.874215 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:12:45.874847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:12:45.875164 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:12:45.876506 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] PATH_INFO: `/v3/projects/7e14a0b0b9524e61a6e90e2cbfcdb9dc` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:12:45.878012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e14a0b0b9524e61a6e90e2cbfcdb9dc)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:12:45.879977 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:12:45.931245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e14a0b0b9524e61a6e90e2cbfcdb9dc', 'request_id': 'req-1b2b1183-807c-46b1-add3-f812e14e97a8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.938628 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e14a0b0b9524e61a6e90e2cbfcdb9dc', 'request_id': 'req-1b2b1183-807c-46b1-add3-f812e14e97a8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.945196 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e14a0b0b9524e61a6e90e2cbfcdb9dc', 'request_id': 'req-1b2b1183-807c-46b1-add3-f812e14e97a8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.955724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] The token cache is being invalidate because project 7e14a0b0b9524e61a6e90e2cbfcdb9dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:12:45.956018 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.956646 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:12:45.958116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1395/2779] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:12:45 2022] DELETE /identity/v3/projects/7e14a0b0b9524e61a6e90e2cbfcdb9dc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:12:55.586850 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:12:55.952538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:02.620537 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63aadbf-e9bd-45d9-b82b-31206f4f6063 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:02.622092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d63aadbf-e9bd-45d9-b82b-31206f4f6063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:02.634223 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63aadbf-e9bd-45d9-b82b-31206f4f6063 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:02.688422 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:02.689179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:02.689455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:02.689741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:02.691484 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:02.776684 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:02.778098 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:02.799318 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1385/2780] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:07.095279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:07.096777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:07.208575 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b1183-807c-46b1-add3-f812e14e97a8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:07.214910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90847b18-4348-41e0-9e16-6798e66f54eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:07.215281 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90847b18-4348-41e0-9e16-6798e66f54eb service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:07.215559 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90847b18-4348-41e0-9e16-6798e66f54eb service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:07.215830 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90847b18-4348-41e0-9e16-6798e66f54eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:07.218740 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-90847b18-4348-41e0-9e16-6798e66f54eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:07.232184 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90847b18-4348-41e0-9e16-6798e66f54eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:07.233881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90847b18-4348-41e0-9e16-6798e66f54eb service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:07.247508 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1396/2781] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:10.267184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:10.268627 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:10.334357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eaca6-ed18-4c85-b900-40ab45a18765 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:10.339204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:10.339770 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:10.340051 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:10.340324 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] PATH_INFO: `/v3/users/26782775a52b48e9a9a84d64d28ceca0` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:10.341459 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=26782775a52b48e9a9a84d64d28ceca0)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:10.343441 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:10.366114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:10.391646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26782775a52b48e9a9a84d64d28ceca0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.399656 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26782775a52b48e9a9a84d64d28ceca0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.411687 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26782775a52b48e9a9a84d64d28ceca0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.427572 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26782775a52b48e9a9a84d64d28ceca0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.428297 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26782775a52b48e9a9a84d64d28ceca0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.435276 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1386/2782] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:13:10 2022] DELETE /identity/v3/users/26782775a52b48e9a9a84d64d28ceca0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:10.607725 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90847b18-4348-41e0-9e16-6798e66f54eb service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:10.619105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90847b18-4348-41e0-9e16-6798e66f54eb service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:10.676877 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:10.677502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:10.677777 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:10.678043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] PATH_INFO: `/v3/projects/cdaf00b54b894a9db2de21ff1ef475a2` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:10.679178 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdaf00b54b894a9db2de21ff1ef475a2)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:10.681072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:10.724623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdaf00b54b894a9db2de21ff1ef475a2', 'request_id': 'req-2821a082-5ff6-4590-bbc3-061850a7a4d9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.731992 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdaf00b54b894a9db2de21ff1ef475a2', 'request_id': 'req-2821a082-5ff6-4590-bbc3-061850a7a4d9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.738181 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdaf00b54b894a9db2de21ff1ef475a2', 'request_id': 'req-2821a082-5ff6-4590-bbc3-061850a7a4d9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.748383 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] The token cache is being invalidate because project cdaf00b54b894a9db2de21ff1ef475a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:13:10.748669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.749287 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.750738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1397/2783] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:13:10 2022] DELETE /identity/v3/projects/cdaf00b54b894a9db2de21ff1ef475a2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:10.754932 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:10.756410 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:10.769944 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3074f3c-82c7-42d1-a4c7-43768bd75f5d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:10.824713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:10.825330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:10.825603 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:10.825870 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] PATH_INFO: `/v3/users/9fa5c90d55214eafa2f00117cd566c92` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:10.827002 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa5c90d55214eafa2f00117cd566c92)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:10.828890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:10.872165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa5c90d55214eafa2f00117cd566c92'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.879716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa5c90d55214eafa2f00117cd566c92'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.886938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa5c90d55214eafa2f00117cd566c92'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.898429 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa5c90d55214eafa2f00117cd566c92'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.899088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa5c90d55214eafa2f00117cd566c92'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:10.904845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1387/2784] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:13:10 2022] DELETE /identity/v3/users/9fa5c90d55214eafa2f00117cd566c92 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:11.083641 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:11.085085 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:11.098428 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a082-5ff6-4590-bbc3-061850a7a4d9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:11.116102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:11.155749 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:11.156304 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:11.156581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:11.156850 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] PATH_INFO: `/v3/projects/0a9e7a325ddc43cd811aba51fe2cc777` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:11.158458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9e7a325ddc43cd811aba51fe2cc777)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:11.160371 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:11.204883 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9e7a325ddc43cd811aba51fe2cc777', 'request_id': 'req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:11.211376 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9e7a325ddc43cd811aba51fe2cc777', 'request_id': 'req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:11.217614 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9e7a325ddc43cd811aba51fe2cc777', 'request_id': 'req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:11.228226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] The token cache is being invalidate because project 0a9e7a325ddc43cd811aba51fe2cc777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:13:11.228503 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:11.229127 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:11.230582 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1398/2785] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:13:11 2022] DELETE /identity/v3/projects/0a9e7a325ddc43cd811aba51fe2cc777 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:13:15.110858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:15.112716 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:15.136472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:15.178525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4a4ac-5ccd-4137-bad6-a2b0a5bf77b5 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:15.227741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:15.228265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:15.228544 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:15.228816 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:15.230692 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:15.305539 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:15.306999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:15.319864 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1388/2786] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:21.225473 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:25.229745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:25.231273 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:25.244567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0811dc-e6dd-42b0-a67e-a713862c3e4d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:25.248226 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:25.248896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:25.249245 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:25.249608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:25.257267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:25.314039 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=65 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:25.316631 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:25.318064 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:25.327783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1399/2787] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:28.413507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:28.416152 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:28.502762 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e900d-802f-4575-8bf9-71a1709d8c68 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:28.508637 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024f986-fcf0-4bc1-904e-11fcd2896186 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:28.509249 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024f986-fcf0-4bc1-904e-11fcd2896186 service placement] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:28.509531 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024f986-fcf0-4bc1-904e-11fcd2896186 service placement] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:28.509825 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024f986-fcf0-4bc1-904e-11fcd2896186 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:28.511859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-b024f986-fcf0-4bc1-904e-11fcd2896186 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:28.524076 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024f986-fcf0-4bc1-904e-11fcd2896186 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:28.525647 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024f986-fcf0-4bc1-904e-11fcd2896186 service placement] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:28.536100 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1389/2788] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:31.389495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:31.391175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:31.404804 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941d588-af11-450c-b6c5-d196c664cccd service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:31.411081 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:31.411831 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:31.412109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:31.412401 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] PATH_INFO: `/v3/users/7a9c654fc9bc4ad4b530d3f4df5c24c8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:31.413816 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a9c654fc9bc4ad4b530d3f4df5c24c8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:31.415490 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:31.464122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a9c654fc9bc4ad4b530d3f4df5c24c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.470805 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a9c654fc9bc4ad4b530d3f4df5c24c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.482991 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a9c654fc9bc4ad4b530d3f4df5c24c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.483655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a9c654fc9bc4ad4b530d3f4df5c24c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.484256 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a9c654fc9bc4ad4b530d3f4df5c24c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.496921 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1400/2789] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:13:31 2022] DELETE /identity/v3/users/7a9c654fc9bc4ad4b530d3f4df5c24c8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:31.671106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024f986-fcf0-4bc1-904e-11fcd2896186 service placement] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:31.682428 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024f986-fcf0-4bc1-904e-11fcd2896186 service placement] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:31.738341 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:31.738984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:31.739264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:31.739534 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] PATH_INFO: `/v3/projects/aa338ea60ddf4f53a71c19d51b21edb8` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:31.740631 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa338ea60ddf4f53a71c19d51b21edb8)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:31.742525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:31.788179 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa338ea60ddf4f53a71c19d51b21edb8', 'request_id': 'req-0cfb1950-ff54-46c8-bd76-649c2977f6c0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.794193 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa338ea60ddf4f53a71c19d51b21edb8', 'request_id': 'req-0cfb1950-ff54-46c8-bd76-649c2977f6c0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.801351 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa338ea60ddf4f53a71c19d51b21edb8', 'request_id': 'req-0cfb1950-ff54-46c8-bd76-649c2977f6c0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.811713 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] The token cache is being invalidate because project aa338ea60ddf4f53a71c19d51b21edb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:13:31.811996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.812615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.814120 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1390/2790] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:13:31 2022] DELETE /identity/v3/projects/aa338ea60ddf4f53a71c19d51b21edb8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:31.818155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:31.819570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:31.833100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34736483-c8cc-48c3-87b1-07f3fc8e84a9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:31.891398 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:31.892027 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:31.892689 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:31.893861 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] PATH_INFO: `/v3/users/61eb6f0791b9440fa355609f4c8fb532` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:31.895012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61eb6f0791b9440fa355609f4c8fb532)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:31.896929 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:31.902053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:31.941912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61eb6f0791b9440fa355609f4c8fb532'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.948588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61eb6f0791b9440fa355609f4c8fb532'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.960548 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61eb6f0791b9440fa355609f4c8fb532'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.961204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61eb6f0791b9440fa355609f4c8fb532'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.961801 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61eb6f0791b9440fa355609f4c8fb532'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:31.974113 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1401/2791] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:13:31 2022] DELETE /identity/v3/users/61eb6f0791b9440fa355609f4c8fb532 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:32.145667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:32.147175 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:32.160436 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb1950-ff54-46c8-bd76-649c2977f6c0 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:32.185399 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:32.227724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:32.228278 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:32.228552 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:32.228818 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] PATH_INFO: `/v3/projects/d56982c987cb47e39c611ea9dddbc6e2` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:32.230034 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56982c987cb47e39c611ea9dddbc6e2)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:32.232041 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:32.276703 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56982c987cb47e39c611ea9dddbc6e2', 'request_id': 'req-1a9cb8f0-c055-4c03-9b78-9455855f6845'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:32.282532 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56982c987cb47e39c611ea9dddbc6e2', 'request_id': 'req-1a9cb8f0-c055-4c03-9b78-9455855f6845'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:32.290425 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56982c987cb47e39c611ea9dddbc6e2', 'request_id': 'req-1a9cb8f0-c055-4c03-9b78-9455855f6845'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:32.300514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] The token cache is being invalidate because project d56982c987cb47e39c611ea9dddbc6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:13:32.300799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:32.301417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:32.302878 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1391/2792] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:13:32 2022] DELETE /identity/v3/projects/d56982c987cb47e39c611ea9dddbc6e2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:41.636005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:41.637664 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:41.750785 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8be841-bcb8-4bca-9377-fbc3f9815c39 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:41.835364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dfcbe-143c-4062-8603-60da8dc01a9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '904f7f036565419b9fdecc5e47acb45c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:41.836103 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dfcbe-143c-4062-8603-60da8dc01a9d service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:41.836378 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dfcbe-143c-4062-8603-60da8dc01a9d service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:41.836653 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dfcbe-143c-4062-8603-60da8dc01a9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:41.838581 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-379dfcbe-143c-4062-8603-60da8dc01a9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:41.852567 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379dfcbe-143c-4062-8603-60da8dc01a9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:41.855538 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379dfcbe-143c-4062-8603-60da8dc01a9d service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:41.873054 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1402/2793] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:42.297586 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:49.726208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:49.727899 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:49.793633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cb8f0-c055-4c03-9b78-9455855f6845 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:49.798142 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:49.798758 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:49.799587 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:49.799859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] PATH_INFO: `/v3/users/67a34ddb337649ee948d319b0bf5fab3` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:49.800948 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67a34ddb337649ee948d319b0bf5fab3)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:49.802836 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:49.847244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67a34ddb337649ee948d319b0bf5fab3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:49.853975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67a34ddb337649ee948d319b0bf5fab3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:49.860699 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67a34ddb337649ee948d319b0bf5fab3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:49.873682 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67a34ddb337649ee948d319b0bf5fab3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:49.874328 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-17d4b9cc-f9d6-48b6-ac24-fd5bd9a683e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67a34ddb337649ee948d319b0bf5fab3'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:49.881075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1392/2794] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:13:49 2022] DELETE /identity/v3/users/67a34ddb337649ee948d319b0bf5fab3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:50.056865 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dfcbe-143c-4062-8603-60da8dc01a9d service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.067902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dfcbe-143c-4062-8603-60da8dc01a9d service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.075159 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:50.132114 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.132667 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.132940 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.133207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] PATH_INFO: `/v3/projects/d006aeb4f532425ab423537ebcb2f0b1` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.134318 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d006aeb4f532425ab423537ebcb2f0b1)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.136243 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.180514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd006aeb4f532425ab423537ebcb2f0b1', 'request_id': 'req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:50.187359 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd006aeb4f532425ab423537ebcb2f0b1', 'request_id': 'req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:50.193223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd006aeb4f532425ab423537ebcb2f0b1', 'request_id': 'req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:50.203151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] The token cache is being invalidate because project d006aeb4f532425ab423537ebcb2f0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:13:50.203440 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:50.204060 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9d0f251-9d8d-464a-a7e4-8a749d4c3531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:50.205522 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1403/2795] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:13:50 2022] DELETE /identity/v3/projects/d006aeb4f532425ab423537ebcb2f0b1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:50.210395 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227388d-8e98-4997-924a-9337dd78492b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.210680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227388d-8e98-4997-924a-9337dd78492b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.210974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227388d-8e98-4997-924a-9337dd78492b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.231563 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-c227388d-8e98-4997-924a-9337dd78492b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:50.232735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-c227388d-8e98-4997-924a-9337dd78492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:50.294310 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1393/2796] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:13:50.303786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce30f5-a83b-44f2-a69b-c1869c8a4a04 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.304068 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce30f5-a83b-44f2-a69b-c1869c8a4a04 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.304388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce30f5-a83b-44f2-a69b-c1869c8a4a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.323340 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a9ce30f5-a83b-44f2-a69b-c1869c8a4a04 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:50.324550 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a9ce30f5-a83b-44f2-a69b-c1869c8a4a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:50.332189 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1404/2797] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:13:50.336252 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227388d-8e98-4997-924a-9337dd78492b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.345507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227388d-8e98-4997-924a-9337dd78492b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.350540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ca528-7c45-4b47-b689-b3f6e2600cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.351147 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ca528-7c45-4b47-b689-b3f6e2600cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.351417 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ca528-7c45-4b47-b689-b3f6e2600cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.351680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ca528-7c45-4b47-b689-b3f6e2600cf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.352308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ca528-7c45-4b47-b689-b3f6e2600cf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:13:50.352604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ca528-7c45-4b47-b689-b3f6e2600cf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.354166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ca528-7c45-4b47-b689-b3f6e2600cf9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.365795 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1394/2798] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:13:50.369962 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce30f5-a83b-44f2-a69b-c1869c8a4a04 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.379316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce30f5-a83b-44f2-a69b-c1869c8a4a04 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.384012 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097ed28-9bb8-4fce-a524-6efaf0e312f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.384624 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097ed28-9bb8-4fce-a524-6efaf0e312f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.384904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097ed28-9bb8-4fce-a524-6efaf0e312f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.385274 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097ed28-9bb8-4fce-a524-6efaf0e312f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.385775 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097ed28-9bb8-4fce-a524-6efaf0e312f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.387771 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097ed28-9bb8-4fce-a524-6efaf0e312f6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.429139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1405/2799] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:13:50 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:13:50.433364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ca528-7c45-4b47-b689-b3f6e2600cf9 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.443348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ca528-7c45-4b47-b689-b3f6e2600cf9 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.446926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:50.500484 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea2e1b-63c4-482e-b609-9aa1bd458bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.501162 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea2e1b-63c4-482e-b609-9aa1bd458bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.501347 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea2e1b-63c4-482e-b609-9aa1bd458bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.501859 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea2e1b-63c4-482e-b609-9aa1bd458bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.502514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea2e1b-63c4-482e-b609-9aa1bd458bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.504735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea2e1b-63c4-482e-b609-9aa1bd458bf0 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.525481 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1395/2800] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:13:50 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:13:50.529709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097ed28-9bb8-4fce-a524-6efaf0e312f6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.540835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097ed28-9bb8-4fce-a524-6efaf0e312f6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.545387 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9de0e-b0ea-4348-ba65-61a7f483a205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.545981 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9de0e-b0ea-4348-ba65-61a7f483a205 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.546262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9de0e-b0ea-4348-ba65-61a7f483a205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.546531 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9de0e-b0ea-4348-ba65-61a7f483a205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.547207 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9de0e-b0ea-4348-ba65-61a7f483a205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.548782 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9de0e-b0ea-4348-ba65-61a7f483a205 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.558533 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1406/2801] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:13:50.562408 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea2e1b-63c4-482e-b609-9aa1bd458bf0 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.572251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea2e1b-63c4-482e-b609-9aa1bd458bf0 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.576867 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9b9bb-0ab1-490b-86ac-547d1633ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.577482 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9b9bb-0ab1-490b-86ac-547d1633ae34 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.577752 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9b9bb-0ab1-490b-86ac-547d1633ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.578018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9b9bb-0ab1-490b-86ac-547d1633ae34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.578646 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9b9bb-0ab1-490b-86ac-547d1633ae34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.580730 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9b9bb-0ab1-490b-86ac-547d1633ae34 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.591037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1396/2802] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:13:50.594896 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9de0e-b0ea-4348-ba65-61a7f483a205 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.605623 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9de0e-b0ea-4348-ba65-61a7f483a205 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.610407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c3cdd-f23e-44cc-a749-a60459876945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.611117 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3cdd-f23e-44cc-a749-a60459876945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.611395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3cdd-f23e-44cc-a749-a60459876945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.611663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3cdd-f23e-44cc-a749-a60459876945 admin admin] PATH_INFO: `/v3/projects/66066781a5f14d398222aae1c92f5db2/users/722ff30934b04821a76c547052d2834b/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.625995 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c3cdd-f23e-44cc-a749-a60459876945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66066781a5f14d398222aae1c92f5db2, user_id=722ff30934b04821a76c547052d2834b, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.628890 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c3cdd-f23e-44cc-a749-a60459876945 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.641702 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1407/2803] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:13:50 2022] PUT /identity/v3/projects/66066781a5f14d398222aae1c92f5db2/users/722ff30934b04821a76c547052d2834b/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:13:50.652283 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11d614-1ffb-4312-a8f2-adaa4a3b4546 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.652565 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11d614-1ffb-4312-a8f2-adaa4a3b4546 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.652833 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11d614-1ffb-4312-a8f2-adaa4a3b4546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.679522 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-6c11d614-1ffb-4312-a8f2-adaa4a3b4546 None None] MFA Rules not processed for user `722ff30934b04821a76c547052d2834b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:50.680699 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-6c11d614-1ffb-4312-a8f2-adaa4a3b4546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:50.743487 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1397/2804] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:13:50.747982 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c3cdd-f23e-44cc-a749-a60459876945 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.777663 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:50.811297 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c3cdd-f23e-44cc-a749-a60459876945 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.815957 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35faf7e3-826b-4c60-8ea6-6bc234ca82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.816557 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faf7e3-826b-4c60-8ea6-6bc234ca82d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.816836 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faf7e3-826b-4c60-8ea6-6bc234ca82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.817113 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faf7e3-826b-4c60-8ea6-6bc234ca82d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.817729 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35faf7e3-826b-4c60-8ea6-6bc234ca82d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.819650 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35faf7e3-826b-4c60-8ea6-6bc234ca82d8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.845407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1408/2805] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:13:50 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:13:50.850127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11d614-1ffb-4312-a8f2-adaa4a3b4546 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.860509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11d614-1ffb-4312-a8f2-adaa4a3b4546 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.915264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e713e-947f-40dc-8938-c978785d671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.915796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e713e-947f-40dc-8938-c978785d671a admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.916072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e713e-947f-40dc-8938-c978785d671a admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.916343 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e713e-947f-40dc-8938-c978785d671a admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.916949 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e713e-947f-40dc-8938-c978785d671a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.918844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e713e-947f-40dc-8938-c978785d671a admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.940698 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1398/2806] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:13:50 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:13:50.944699 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35faf7e3-826b-4c60-8ea6-6bc234ca82d8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.954644 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35faf7e3-826b-4c60-8ea6-6bc234ca82d8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.959786 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96584ba-f429-4a8a-85f0-1ee979611eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.960373 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96584ba-f429-4a8a-85f0-1ee979611eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.960648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96584ba-f429-4a8a-85f0-1ee979611eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.960914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96584ba-f429-4a8a-85f0-1ee979611eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:50.961544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96584ba-f429-4a8a-85f0-1ee979611eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:50.963151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96584ba-f429-4a8a-85f0-1ee979611eb2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:50.973407 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1409/2807] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:13:50.977398 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e713e-947f-40dc-8938-c978785d671a admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:50.983659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:50.987956 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e713e-947f-40dc-8938-c978785d671a admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:50.993965 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae8f8-e1dc-4721-a01a-76b4f43e1b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:50.994632 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae8f8-e1dc-4721-a01a-76b4f43e1b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:50.994923 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae8f8-e1dc-4721-a01a-76b4f43e1b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:50.995199 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae8f8-e1dc-4721-a01a-76b4f43e1b9d admin admin] PATH_INFO: `/v3/projects/9dd464df773649b5933ba33885883aff/users/ab6e1ea78bbe423c8c2717cdb52c54d0/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:51.009983 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cae8f8-e1dc-4721-a01a-76b4f43e1b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd464df773649b5933ba33885883aff, user_id=ab6e1ea78bbe423c8c2717cdb52c54d0, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:51.012645 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cae8f8-e1dc-4721-a01a-76b4f43e1b9d admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:51.025312 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1399/2808] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:13:50 2022] PUT /identity/v3/projects/9dd464df773649b5933ba33885883aff/users/ab6e1ea78bbe423c8c2717cdb52c54d0/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:51.029219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96584ba-f429-4a8a-85f0-1ee979611eb2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:51.038800 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96584ba-f429-4a8a-85f0-1ee979611eb2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:51.092605 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e33adb-de6f-4aef-a593-40a5e8130ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:51.093195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e33adb-de6f-4aef-a593-40a5e8130ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:51.093456 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e33adb-de6f-4aef-a593-40a5e8130ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:51.093724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e33adb-de6f-4aef-a593-40a5e8130ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:51.094354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e33adb-de6f-4aef-a593-40a5e8130ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:51.095972 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e33adb-de6f-4aef-a593-40a5e8130ffe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:51.105336 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1410/2809] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 10:13:51.109611 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae8f8-e1dc-4721-a01a-76b4f43e1b9d admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:51.119699 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae8f8-e1dc-4721-a01a-76b4f43e1b9d admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:51.124776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d551e-5c7e-4ae2-aa20-d050afb94ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:51.125415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d551e-5c7e-4ae2-aa20-d050afb94ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:51.125689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d551e-5c7e-4ae2-aa20-d050afb94ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:51.125960 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d551e-5c7e-4ae2-aa20-d050afb94ed9 admin admin] PATH_INFO: `/v3/projects/9dd464df773649b5933ba33885883aff/users/ab6e1ea78bbe423c8c2717cdb52c54d0/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:51.128607 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d551e-5c7e-4ae2-aa20-d050afb94ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd464df773649b5933ba33885883aff, user_id=ab6e1ea78bbe423c8c2717cdb52c54d0, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:51.131244 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d551e-5c7e-4ae2-aa20-d050afb94ed9 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:51.141158 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1400/2810] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:13:51 2022] PUT /identity/v3/projects/9dd464df773649b5933ba33885883aff/users/ab6e1ea78bbe423c8c2717cdb52c54d0/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:13:51.153100 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224b79d-8bdf-4412-a6ef-949a582ccf7c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:51.153382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224b79d-8bdf-4412-a6ef-949a582ccf7c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:51.153679 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224b79d-8bdf-4412-a6ef-949a582ccf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:51.180276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2224b79d-8bdf-4412-a6ef-949a582ccf7c None None] MFA Rules not processed for user `ab6e1ea78bbe423c8c2717cdb52c54d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:51.181395 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2224b79d-8bdf-4412-a6ef-949a582ccf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:51.246912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1411/2811] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:13:51.258492 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d551e-5c7e-4ae2-aa20-d050afb94ed9 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:51.260208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-f53d551e-5c7e-4ae2-aa20-d050afb94ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:51.317190 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d551e-5c7e-4ae2-aa20-d050afb94ed9 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:51.320883 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f946bec-b701-48e3-9074-f39e074679eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:51.321480 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f946bec-b701-48e3-9074-f39e074679eb service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:51.321754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f946bec-b701-48e3-9074-f39e074679eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:51.322018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f946bec-b701-48e3-9074-f39e074679eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:51.330778 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f946bec-b701-48e3-9074-f39e074679eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:51.332217 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f946bec-b701-48e3-9074-f39e074679eb service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:51.341259 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1401/2812] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:51.443165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224b79d-8bdf-4412-a6ef-949a582ccf7c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:51.452597 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224b79d-8bdf-4412-a6ef-949a582ccf7c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:51.456116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1053e6-91f9-4b4b-8be2-9148e62b6d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:51.456700 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1053e6-91f9-4b4b-8be2-9148e62b6d3b service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:51.456974 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1053e6-91f9-4b4b-8be2-9148e62b6d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:51.457238 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1053e6-91f9-4b4b-8be2-9148e62b6d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:51.475671 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:51.514092 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1053e6-91f9-4b4b-8be2-9148e62b6d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:51.515555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1053e6-91f9-4b4b-8be2-9148e62b6d3b service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:51.524338 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1412/2813] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:54.101393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f946bec-b701-48e3-9074-f39e074679eb service cinder] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:54.161568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f946bec-b701-48e3-9074-f39e074679eb service cinder] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:54.166018 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:54.166615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:54.166888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:54.167180 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] PATH_INFO: `/v3/users/722ff30934b04821a76c547052d2834b` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:54.168260 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=722ff30934b04821a76c547052d2834b)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:54.170141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:54.214840 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '722ff30934b04821a76c547052d2834b'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.221804 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '722ff30934b04821a76c547052d2834b'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.228634 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '722ff30934b04821a76c547052d2834b'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.235568 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:54.242204 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '722ff30934b04821a76c547052d2834b'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.242888 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '722ff30934b04821a76c547052d2834b'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.248779 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1402/2814] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:13:54 2022] DELETE /identity/v3/users/722ff30934b04821a76c547052d2834b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:54.254237 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ab8dc-ba0a-4d18-b0ed-5e0a2c023a5f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:54.254708 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ab8dc-ba0a-4d18-b0ed-5e0a2c023a5f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:54.254815 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ab8dc-ba0a-4d18-b0ed-5e0a2c023a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:54.273759 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5c0ab8dc-ba0a-4d18-b0ed-5e0a2c023a5f None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:54.274984 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5c0ab8dc-ba0a-4d18-b0ed-5e0a2c023a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:54.335878 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1413/2815] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:13:54.345420 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:54.346832 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:54.402765 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071bf66-b97e-4a18-b78e-0d0a2f33c145 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:54.406407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccf9b5-7903-44bd-b0cb-c911ff903bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:54.407031 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccf9b5-7903-44bd-b0cb-c911ff903bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:54.407309 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccf9b5-7903-44bd-b0cb-c911ff903bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:54.407581 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccf9b5-7903-44bd-b0cb-c911ff903bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:54.416605 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccf9b5-7903-44bd-b0cb-c911ff903bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:54.418020 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccf9b5-7903-44bd-b0cb-c911ff903bd4 service neutron] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:54.427751 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1403/2816] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:54.697677 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ab8dc-ba0a-4d18-b0ed-5e0a2c023a5f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:54.699148 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5c0ab8dc-ba0a-4d18-b0ed-5e0a2c023a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:54.711354 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ab8dc-ba0a-4d18-b0ed-5e0a2c023a5f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:54.715904 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:54.716666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:54.716946 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:54.717212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] PATH_INFO: `/v3/projects/66066781a5f14d398222aae1c92f5db2` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:54.718430 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66066781a5f14d398222aae1c92f5db2)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:54.720670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:54.766570 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66066781a5f14d398222aae1c92f5db2', 'request_id': 'req-07fdd097-c2b8-4c45-813d-c27c2da14441'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.773454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66066781a5f14d398222aae1c92f5db2', 'request_id': 'req-07fdd097-c2b8-4c45-813d-c27c2da14441'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.779760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66066781a5f14d398222aae1c92f5db2', 'request_id': 'req-07fdd097-c2b8-4c45-813d-c27c2da14441'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.790388 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] The token cache is being invalidate because project 66066781a5f14d398222aae1c92f5db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:13:54.790670 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.791334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.792798 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1414/2817] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:13:54 2022] DELETE /identity/v3/projects/66066781a5f14d398222aae1c92f5db2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:54.796914 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccf9b5-7903-44bd-b0cb-c911ff903bd4 service neutron] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:54.807124 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccf9b5-7903-44bd-b0cb-c911ff903bd4 service neutron] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:54.860488 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:54.861026 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:54.861302 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:54.862181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] PATH_INFO: `/v3/users/ab6e1ea78bbe423c8c2717cdb52c54d0` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:54.863311 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab6e1ea78bbe423c8c2717cdb52c54d0)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:54.865206 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:54.909553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab6e1ea78bbe423c8c2717cdb52c54d0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.916445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab6e1ea78bbe423c8c2717cdb52c54d0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.923265 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab6e1ea78bbe423c8c2717cdb52c54d0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.935507 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab6e1ea78bbe423c8c2717cdb52c54d0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.936149 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b4d12ae1-df10-4b36-8575-83310072ae55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab6e1ea78bbe423c8c2717cdb52c54d0'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:54.942188 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1404/2818] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:13:54 2022] DELETE /identity/v3/users/ab6e1ea78bbe423c8c2717cdb52c54d0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:55.206104 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:55.207638 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:55.220588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdd097-c2b8-4c45-813d-c27c2da14441 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:55.262504 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:55.279294 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:55.279932 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.280212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.280483 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] PATH_INFO: `/v3/projects/9dd464df773649b5933ba33885883aff` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.281596 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd464df773649b5933ba33885883aff)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:55.284225 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:55.328965 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd464df773649b5933ba33885883aff', 'request_id': 'req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:55.336067 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd464df773649b5933ba33885883aff', 'request_id': 'req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:55.342358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd464df773649b5933ba33885883aff', 'request_id': 'req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:55.353045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] The token cache is being invalidate because project 9dd464df773649b5933ba33885883aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:13:55.353331 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:55.353953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bcd9dad0-ddd7-45df-8362-7ec3428d7d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:55.355449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1415/2819] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:13:55 2022] DELETE /identity/v3/projects/9dd464df773649b5933ba33885883aff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:55.360255 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c3029-2d20-41ca-8436-f7834e6428f3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.360733 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c3029-2d20-41ca-8436-f7834e6428f3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.360844 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c3029-2d20-41ca-8436-f7834e6428f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.380357 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-e79c3029-2d20-41ca-8436-f7834e6428f3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:55.381576 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-e79c3029-2d20-41ca-8436-f7834e6428f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:55.386541 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:55.440490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1405/2820] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:13:55.448279 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e95e0-7e73-4850-8196-4e323187f143 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.448565 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e95e0-7e73-4850-8196-4e323187f143 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.448837 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e95e0-7e73-4850-8196-4e323187f143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.468770 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3b6e95e0-7e73-4850-8196-4e323187f143 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:55.469947 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3b6e95e0-7e73-4850-8196-4e323187f143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:55.477626 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1416/2821] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:13:55.481807 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c3029-2d20-41ca-8436-f7834e6428f3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:55.491438 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c3029-2d20-41ca-8436-f7834e6428f3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:55.496042 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf087b-ee51-4bdc-96cc-c7b1dc64a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:55.496633 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf087b-ee51-4bdc-96cc-c7b1dc64a6da admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.496912 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf087b-ee51-4bdc-96cc-c7b1dc64a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.497185 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf087b-ee51-4bdc-96cc-c7b1dc64a6da admin admin] PATH_INFO: `/v3/domains` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.497826 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf087b-ee51-4bdc-96cc-c7b1dc64a6da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80246) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:13:55.498127 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf087b-ee51-4bdc-96cc-c7b1dc64a6da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:55.499743 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf087b-ee51-4bdc-96cc-c7b1dc64a6da admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:55.511754 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1406/2822] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:13:55.515860 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e95e0-7e73-4850-8196-4e323187f143 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:55.525674 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e95e0-7e73-4850-8196-4e323187f143 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:55.530510 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441ae78-2983-4e57-b384-5c75687416fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:55.531139 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441ae78-2983-4e57-b384-5c75687416fb admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.531422 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441ae78-2983-4e57-b384-5c75687416fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.531694 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441ae78-2983-4e57-b384-5c75687416fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.532351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441ae78-2983-4e57-b384-5c75687416fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:55.534235 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441ae78-2983-4e57-b384-5c75687416fb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:55.560199 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1417/2823] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:13:55 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:13:55.564174 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf087b-ee51-4bdc-96cc-c7b1dc64a6da admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:55.574282 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf087b-ee51-4bdc-96cc-c7b1dc64a6da admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:55.629710 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5996631-7b5a-4346-ac62-0c78d7c5224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:55.630330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5996631-7b5a-4346-ac62-0c78d7c5224f admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.630584 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5996631-7b5a-4346-ac62-0c78d7c5224f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.630871 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5996631-7b5a-4346-ac62-0c78d7c5224f admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.631560 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5996631-7b5a-4346-ac62-0c78d7c5224f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:55.633521 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5996631-7b5a-4346-ac62-0c78d7c5224f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:55.653933 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1407/2824] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:13:55 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:13:55.658116 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441ae78-2983-4e57-b384-5c75687416fb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:55.668119 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441ae78-2983-4e57-b384-5c75687416fb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:55.672732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d116d8-616b-41e5-876b-77552aa3dc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:55.673331 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d116d8-616b-41e5-876b-77552aa3dc3f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.673608 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d116d8-616b-41e5-876b-77552aa3dc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.673881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d116d8-616b-41e5-876b-77552aa3dc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.674517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d116d8-616b-41e5-876b-77552aa3dc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:55.676109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d116d8-616b-41e5-876b-77552aa3dc3f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:55.686154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1418/2825] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:13:55.690329 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5996631-7b5a-4346-ac62-0c78d7c5224f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:55.699942 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5996631-7b5a-4346-ac62-0c78d7c5224f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:55.704533 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ee728-c2b3-4f0a-a9d5-5f33d93a4eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:55.705112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ee728-c2b3-4f0a-a9d5-5f33d93a4eae admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.705942 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ee728-c2b3-4f0a-a9d5-5f33d93a4eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.706364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ee728-c2b3-4f0a-a9d5-5f33d93a4eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.707232 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ee728-c2b3-4f0a-a9d5-5f33d93a4eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:55.708540 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ee728-c2b3-4f0a-a9d5-5f33d93a4eae admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:55.719066 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1408/2826] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:13:55.725122 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d116d8-616b-41e5-876b-77552aa3dc3f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:55.733136 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d116d8-616b-41e5-876b-77552aa3dc3f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:55.738212 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdfa1b-6953-4ee3-b7ac-ae72fc9a28a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:55.738888 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdfa1b-6953-4ee3-b7ac-ae72fc9a28a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.739204 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdfa1b-6953-4ee3-b7ac-ae72fc9a28a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.739485 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdfa1b-6953-4ee3-b7ac-ae72fc9a28a4 admin admin] PATH_INFO: `/v3/projects/ff7c7f3f5c2a4fb1b12addf87821de92/users/5f8c939301574077ababe08263837e8b/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.755327 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfdfa1b-6953-4ee3-b7ac-ae72fc9a28a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7c7f3f5c2a4fb1b12addf87821de92, user_id=5f8c939301574077ababe08263837e8b, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:55.757964 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfdfa1b-6953-4ee3-b7ac-ae72fc9a28a4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:55.770477 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1419/2827] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:13:55 2022] PUT /identity/v3/projects/ff7c7f3f5c2a4fb1b12addf87821de92/users/5f8c939301574077ababe08263837e8b/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 10:13:55.780711 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb74a90-da8d-4407-95e8-ac5c8e95960b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.780995 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb74a90-da8d-4407-95e8-ac5c8e95960b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.781270 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb74a90-da8d-4407-95e8-ac5c8e95960b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.808050 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-7cb74a90-da8d-4407-95e8-ac5c8e95960b None None] MFA Rules not processed for user `5f8c939301574077ababe08263837e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:55.809234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-7cb74a90-da8d-4407-95e8-ac5c8e95960b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:55.872014 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1409/2828] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:13:55.876565 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdfa1b-6953-4ee3-b7ac-ae72fc9a28a4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:55.937325 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdfa1b-6953-4ee3-b7ac-ae72fc9a28a4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:55.942458 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27314397-01c5-4d50-bb25-6960d8086b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:55.943082 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27314397-01c5-4d50-bb25-6960d8086b37 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:55.943361 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27314397-01c5-4d50-bb25-6960d8086b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:55.943629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27314397-01c5-4d50-bb25-6960d8086b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:55.944234 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27314397-01c5-4d50-bb25-6960d8086b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:55.946105 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27314397-01c5-4d50-bb25-6960d8086b37 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:55.960543 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:55.974528 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1420/2829] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:13:55 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:13:55.978529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb74a90-da8d-4407-95e8-ac5c8e95960b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:55.989885 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb74a90-da8d-4407-95e8-ac5c8e95960b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:55.995184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:56.048573 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33601d-b4e6-448c-ab29-765d050e6469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:56.049662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33601d-b4e6-448c-ab29-765d050e6469 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:56.049940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33601d-b4e6-448c-ab29-765d050e6469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:56.050215 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33601d-b4e6-448c-ab29-765d050e6469 admin admin] PATH_INFO: `/v3/users` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:56.050845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33601d-b4e6-448c-ab29-765d050e6469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:56.052792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33601d-b4e6-448c-ab29-765d050e6469 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:56.074445 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1410/2830] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:13:55 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:13:56.078382 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27314397-01c5-4d50-bb25-6960d8086b37 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:56.088732 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27314397-01c5-4d50-bb25-6960d8086b37 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:56.093314 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572deb6-7105-4afa-a0e1-9c5b48672717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:56.093909 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572deb6-7105-4afa-a0e1-9c5b48672717 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:56.094193 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572deb6-7105-4afa-a0e1-9c5b48672717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:56.094468 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572deb6-7105-4afa-a0e1-9c5b48672717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:56.095154 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572deb6-7105-4afa-a0e1-9c5b48672717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:56.097271 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572deb6-7105-4afa-a0e1-9c5b48672717 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:56.107845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1421/2831] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:13:56.112449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33601d-b4e6-448c-ab29-765d050e6469 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:56.122088 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33601d-b4e6-448c-ab29-765d050e6469 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:56.126585 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4901b-5907-4240-bbf9-9f34b41f6a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:56.127502 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4901b-5907-4240-bbf9-9f34b41f6a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:56.127776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4901b-5907-4240-bbf9-9f34b41f6a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:56.128047 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4901b-5907-4240-bbf9-9f34b41f6a52 admin admin] PATH_INFO: `/v3/projects/ad5c9352e79042edbdec34446fd2b42a/users/2a8fc4296d60413b88a40a4d79615651/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:56.143460 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4901b-5907-4240-bbf9-9f34b41f6a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5c9352e79042edbdec34446fd2b42a, user_id=2a8fc4296d60413b88a40a4d79615651, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:56.146075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4901b-5907-4240-bbf9-9f34b41f6a52 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:56.158874 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1411/2832] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:13:56 2022] PUT /identity/v3/projects/ad5c9352e79042edbdec34446fd2b42a/users/2a8fc4296d60413b88a40a4d79615651/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:56.162634 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572deb6-7105-4afa-a0e1-9c5b48672717 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:56.172554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572deb6-7105-4afa-a0e1-9c5b48672717 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:56.226454 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0fbe2-8327-4352-97d0-316d29f0f66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:56.226997 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0fbe2-8327-4352-97d0-316d29f0f66a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:56.227276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0fbe2-8327-4352-97d0-316d29f0f66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:56.227544 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0fbe2-8327-4352-97d0-316d29f0f66a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:56.228165 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0fbe2-8327-4352-97d0-316d29f0f66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:56.229758 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0fbe2-8327-4352-97d0-316d29f0f66a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:56.239825 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1422/2833] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:13:56.243935 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4901b-5907-4240-bbf9-9f34b41f6a52 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:56.254037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4901b-5907-4240-bbf9-9f34b41f6a52 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:56.258523 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac710490-79a1-4f3f-8a05-92ce756b4abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:56.259177 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac710490-79a1-4f3f-8a05-92ce756b4abf admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:56.259452 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac710490-79a1-4f3f-8a05-92ce756b4abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:56.259719 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac710490-79a1-4f3f-8a05-92ce756b4abf admin admin] PATH_INFO: `/v3/projects/ad5c9352e79042edbdec34446fd2b42a/users/2a8fc4296d60413b88a40a4d79615651/roles/8417343e9ef64991bcf51bb9692da102` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:56.261567 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac710490-79a1-4f3f-8a05-92ce756b4abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5c9352e79042edbdec34446fd2b42a, user_id=2a8fc4296d60413b88a40a4d79615651, role_id=8417343e9ef64991bcf51bb9692da102)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:56.264196 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac710490-79a1-4f3f-8a05-92ce756b4abf admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:56.273783 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1412/2834] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:13:56 2022] PUT /identity/v3/projects/ad5c9352e79042edbdec34446fd2b42a/users/2a8fc4296d60413b88a40a4d79615651/roles/8417343e9ef64991bcf51bb9692da102 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:56.285155 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528ab5c-fa2a-493c-8806-b465ef1bbc7d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:56.286099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528ab5c-fa2a-493c-8806-b465ef1bbc7d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:56.286099 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528ab5c-fa2a-493c-8806-b465ef1bbc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:56.313672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b528ab5c-fa2a-493c-8806-b465ef1bbc7d None None] MFA Rules not processed for user `2a8fc4296d60413b88a40a4d79615651`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:56.314879 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b528ab5c-fa2a-493c-8806-b465ef1bbc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:56.380183 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1423/2835] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:13:56.390901 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac710490-79a1-4f3f-8a05-92ce756b4abf admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:56.392727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ac710490-79a1-4f3f-8a05-92ce756b4abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:56.448955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac710490-79a1-4f3f-8a05-92ce756b4abf admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:56.452595 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b6c37-ff8e-49e7-b42e-9357c40b9500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:56.453184 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b6c37-ff8e-49e7-b42e-9357c40b9500 service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:56.453455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b6c37-ff8e-49e7-b42e-9357c40b9500 service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:56.453722 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b6c37-ff8e-49e7-b42e-9357c40b9500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:56.462974 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b6c37-ff8e-49e7-b42e-9357c40b9500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:56.464387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b6c37-ff8e-49e7-b42e-9357c40b9500 service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:56.474166 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1413/2836] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:56.838358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528ab5c-fa2a-493c-8806-b465ef1bbc7d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:56.900240 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528ab5c-fa2a-493c-8806-b465ef1bbc7d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:56.904683 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:56.905296 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:56.905578 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:56.905849 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] PATH_INFO: `/v3/users/5f8c939301574077ababe08263837e8b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:56.906978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8c939301574077ababe08263837e8b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:56.908912 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:56.914087 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:56.954817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8c939301574077ababe08263837e8b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:56.962754 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8c939301574077ababe08263837e8b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:56.976213 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8c939301574077ababe08263837e8b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:56.976934 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8c939301574077ababe08263837e8b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:56.977564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8c939301574077ababe08263837e8b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:56.989987 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1424/2837] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:13:56 2022] DELETE /identity/v3/users/5f8c939301574077ababe08263837e8b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:56.994543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c453a-bb9e-4a9c-9d53-a64286bf368d None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:56.994827 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c453a-bb9e-4a9c-9d53-a64286bf368d None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:56.995144 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c453a-bb9e-4a9c-9d53-a64286bf368d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:57.013922 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-ba9c453a-bb9e-4a9c-9d53-a64286bf368d None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:57.015141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ba9c453a-bb9e-4a9c-9d53-a64286bf368d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:57.072101 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1414/2838] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:13:57.082990 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:57.084374 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:57.139512 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9468d78-8ed8-47b2-80c4-bc017caa6169 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:57.143835 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b3cc-bc1f-42e5-90ed-278634f5bf9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:57.144472 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b3cc-bc1f-42e5-90ed-278634f5bf9b service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:57.144753 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b3cc-bc1f-42e5-90ed-278634f5bf9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:57.145026 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b3cc-bc1f-42e5-90ed-278634f5bf9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:57.153978 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519b3cc-bc1f-42e5-90ed-278634f5bf9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:57.155429 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519b3cc-bc1f-42e5-90ed-278634f5bf9b service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:57.164643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1425/2839] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:13:57.431141 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c453a-bb9e-4a9c-9d53-a64286bf368d None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:57.432764 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-ba9c453a-bb9e-4a9c-9d53-a64286bf368d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:57.440272 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:57.447789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c453a-bb9e-4a9c-9d53-a64286bf368d None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:57.452792 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:57.453415 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:57.453695 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:57.453962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] PATH_INFO: `/v3/projects/ff7c7f3f5c2a4fb1b12addf87821de92` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:57.455242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7c7f3f5c2a4fb1b12addf87821de92)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:57.456996 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:57.501463 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7c7f3f5c2a4fb1b12addf87821de92', 'request_id': 'req-101ff68d-ee9c-477a-b570-05d24c341a93'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.508308 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7c7f3f5c2a4fb1b12addf87821de92', 'request_id': 'req-101ff68d-ee9c-477a-b570-05d24c341a93'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.515221 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7c7f3f5c2a4fb1b12addf87821de92', 'request_id': 'req-101ff68d-ee9c-477a-b570-05d24c341a93'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.525275 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] The token cache is being invalidate because project ff7c7f3f5c2a4fb1b12addf87821de92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:13:57.525558 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.526171 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.527664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1415/2840] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:13:57 2022] DELETE /identity/v3/projects/ff7c7f3f5c2a4fb1b12addf87821de92 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:57.531709 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b3cc-bc1f-42e5-90ed-278634f5bf9b service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:57.541791 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b3cc-bc1f-42e5-90ed-278634f5bf9b service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:57.597942 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:57.598583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:57.598885 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:57.599190 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] PATH_INFO: `/v3/users/2a8fc4296d60413b88a40a4d79615651` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:57.600341 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8fc4296d60413b88a40a4d79615651)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:57.603517 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:57.637962 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:57.652365 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8fc4296d60413b88a40a4d79615651'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.659594 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8fc4296d60413b88a40a4d79615651'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.671005 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8fc4296d60413b88a40a4d79615651'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.671672 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8fc4296d60413b88a40a4d79615651'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.672276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c4a0bf9b-b098-401b-a684-694a7dcbd6b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8fc4296d60413b88a40a4d79615651'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:57.684844 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1426/2841] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:13:57 2022] DELETE /identity/v3/users/2a8fc4296d60413b88a40a4d79615651 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:57.947340 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:57.948845 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:57.965677 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ff68d-ee9c-477a-b570-05d24c341a93 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:58.022971 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:58.023598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.023872 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.024140 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] PATH_INFO: `/v3/projects/ad5c9352e79042edbdec34446fd2b42a` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.025234 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad5c9352e79042edbdec34446fd2b42a)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:58.027209 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:58.071472 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad5c9352e79042edbdec34446fd2b42a', 'request_id': 'req-090e0be2-1f5c-4647-bd20-315d36e4d7d7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:58.077363 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad5c9352e79042edbdec34446fd2b42a', 'request_id': 'req-090e0be2-1f5c-4647-bd20-315d36e4d7d7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:58.084958 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad5c9352e79042edbdec34446fd2b42a', 'request_id': 'req-090e0be2-1f5c-4647-bd20-315d36e4d7d7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:58.094809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] The token cache is being invalidate because project ad5c9352e79042edbdec34446fd2b42a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:13:58.095122 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:58.095735 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-090e0be2-1f5c-4647-bd20-315d36e4d7d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:13:58.097728 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1416/2842] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:13:57 2022] DELETE /identity/v3/projects/ad5c9352e79042edbdec34446fd2b42a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:58.102619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65290d-05cb-49cb-bb40-69e8a3c07ce3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.102906 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65290d-05cb-49cb-bb40-69e8a3c07ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.103246 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65290d-05cb-49cb-bb40-69e8a3c07ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.122009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2b65290d-05cb-49cb-bb40-69e8a3c07ce3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:58.123218 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2b65290d-05cb-49cb-bb40-69e8a3c07ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:58.179823 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1427/2843] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:13:58.187364 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eda5c6-d0f8-496c-9564-fd05eea1a956 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.187667 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eda5c6-d0f8-496c-9564-fd05eea1a956 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.187951 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eda5c6-d0f8-496c-9564-fd05eea1a956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.207622 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-97eda5c6-d0f8-496c-9564-fd05eea1a956 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:58.208823 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-97eda5c6-d0f8-496c-9564-fd05eea1a956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:58.216696 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1417/2844] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:13:58.220693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65290d-05cb-49cb-bb40-69e8a3c07ce3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:58.230859 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65290d-05cb-49cb-bb40-69e8a3c07ce3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:58.235514 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807a534-04e4-4219-bdb7-8ecf09bf54c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:58.236109 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807a534-04e4-4219-bdb7-8ecf09bf54c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.236392 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807a534-04e4-4219-bdb7-8ecf09bf54c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.236666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807a534-04e4-4219-bdb7-8ecf09bf54c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.237310 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807a534-04e4-4219-bdb7-8ecf09bf54c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:13:58.237611 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807a534-04e4-4219-bdb7-8ecf09bf54c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:58.239254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807a534-04e4-4219-bdb7-8ecf09bf54c8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:58.250438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1428/2845] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:13:58.254709 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eda5c6-d0f8-496c-9564-fd05eea1a956 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:58.263689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eda5c6-d0f8-496c-9564-fd05eea1a956 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:58.268843 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073eb898-17db-497e-bc76-6e209d12e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:58.269426 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073eb898-17db-497e-bc76-6e209d12e5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.269702 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073eb898-17db-497e-bc76-6e209d12e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.269972 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073eb898-17db-497e-bc76-6e209d12e5d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.270591 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073eb898-17db-497e-bc76-6e209d12e5d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:58.272529 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073eb898-17db-497e-bc76-6e209d12e5d5 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:58.277785 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:58.301604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1418/2846] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:13:58 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:13:58.305802 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807a534-04e4-4219-bdb7-8ecf09bf54c8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:58.315475 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807a534-04e4-4219-bdb7-8ecf09bf54c8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:58.370437 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a571374-73da-4719-b6b3-7e5a574f1c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:58.371055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a571374-73da-4719-b6b3-7e5a574f1c11 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.371334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a571374-73da-4719-b6b3-7e5a574f1c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.371606 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a571374-73da-4719-b6b3-7e5a574f1c11 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.372227 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a571374-73da-4719-b6b3-7e5a574f1c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:58.374120 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a571374-73da-4719-b6b3-7e5a574f1c11 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:58.396251 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1429/2847] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:13:58 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:13:58.399881 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073eb898-17db-497e-bc76-6e209d12e5d5 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:58.410155 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073eb898-17db-497e-bc76-6e209d12e5d5 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:58.414720 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727b3e4-948b-48ac-b7f8-787102ee3960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:58.415338 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727b3e4-948b-48ac-b7f8-787102ee3960 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.415662 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727b3e4-948b-48ac-b7f8-787102ee3960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.415931 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727b3e4-948b-48ac-b7f8-787102ee3960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.416557 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727b3e4-948b-48ac-b7f8-787102ee3960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:58.418121 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727b3e4-948b-48ac-b7f8-787102ee3960 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:58.428379 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1419/2848] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:13:58.432386 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a571374-73da-4719-b6b3-7e5a574f1c11 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:58.443055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a571374-73da-4719-b6b3-7e5a574f1c11 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:58.447602 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792468b-c3a5-413e-abdf-19c836b5cf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:58.448187 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792468b-c3a5-413e-abdf-19c836b5cf41 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.448459 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792468b-c3a5-413e-abdf-19c836b5cf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.448736 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792468b-c3a5-413e-abdf-19c836b5cf41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.449369 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792468b-c3a5-413e-abdf-19c836b5cf41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:58.450964 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792468b-c3a5-413e-abdf-19c836b5cf41 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:58.461757 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1430/2849] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:13:58.465657 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727b3e4-948b-48ac-b7f8-787102ee3960 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:58.475407 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727b3e4-948b-48ac-b7f8-787102ee3960 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:58.479967 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad7547-ef2c-4935-8d79-22bc6ef7c843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:58.480615 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad7547-ef2c-4935-8d79-22bc6ef7c843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.480887 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad7547-ef2c-4935-8d79-22bc6ef7c843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.481157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad7547-ef2c-4935-8d79-22bc6ef7c843 admin admin] PATH_INFO: `/v3/projects/bc50ec3836b94cca8230d876191e360f/users/14f52e2b94be4ae18ae419ecf3f9bbdc/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.495053 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad7547-ef2c-4935-8d79-22bc6ef7c843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc50ec3836b94cca8230d876191e360f, user_id=14f52e2b94be4ae18ae419ecf3f9bbdc, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:58.497658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad7547-ef2c-4935-8d79-22bc6ef7c843 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:58.510680 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1420/2850] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:13:58 2022] PUT /identity/v3/projects/bc50ec3836b94cca8230d876191e360f/users/14f52e2b94be4ae18ae419ecf3f9bbdc/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:13:58.521267 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0fde7-38ee-471b-b540-93bcf0cbe64c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.521574 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0fde7-38ee-471b-b540-93bcf0cbe64c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.521857 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0fde7-38ee-471b-b540-93bcf0cbe64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.546248 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:13:58.549555 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4aa0fde7-38ee-471b-b540-93bcf0cbe64c None None] MFA Rules not processed for user `14f52e2b94be4ae18ae419ecf3f9bbdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:13:58.551151 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4aa0fde7-38ee-471b-b540-93bcf0cbe64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:58.614223 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1431/2851] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:13:58.625442 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad7547-ef2c-4935-8d79-22bc6ef7c843 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:13:58.627993 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-d6ad7547-ef2c-4935-8d79-22bc6ef7c843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:13:58.683688 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad7547-ef2c-4935-8d79-22bc6ef7c843 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:13:58.687342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da071b-0d90-474c-b98b-04a69b44ac1d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8f456f0d15b4d068cfcd6476563247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:13:58.687938 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da071b-0d90-474c-b98b-04a69b44ac1d service glance] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:13:58.688216 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da071b-0d90-474c-b98b-04a69b44ac1d service glance] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:13:58.688489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da071b-0d90-474c-b98b-04a69b44ac1d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:13:58.697044 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da071b-0d90-474c-b98b-04a69b44ac1d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:13:58.698464 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da071b-0d90-474c-b98b-04a69b44ac1d service glance] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:13:58.707831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1421/2852] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:13:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:14:08.611364 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=26 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:14:08.703279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:14:14.336971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0fde7-38ee-471b-b540-93bcf0cbe64c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:14.398252 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0fde7-38ee-471b-b540-93bcf0cbe64c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:14.402710 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:14.403370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:14.403648 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:14.403914 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] PATH_INFO: `/v3/users/14f52e2b94be4ae18ae419ecf3f9bbdc` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:14.405045 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14f52e2b94be4ae18ae419ecf3f9bbdc)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:14.407015 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:14.454030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14f52e2b94be4ae18ae419ecf3f9bbdc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:14.461219 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14f52e2b94be4ae18ae419ecf3f9bbdc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:14.473293 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14f52e2b94be4ae18ae419ecf3f9bbdc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:14.473961 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14f52e2b94be4ae18ae419ecf3f9bbdc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:14.474588 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14f52e2b94be4ae18ae419ecf3f9bbdc'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:14.486866 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1432/2853] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:14:14 2022] DELETE /identity/v3/users/14f52e2b94be4ae18ae419ecf3f9bbdc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:14:14.491509 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35048c-e348-4e50-8588-35fbee8d4df3 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:14.491980 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35048c-e348-4e50-8588-35fbee8d4df3 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:14.492092 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35048c-e348-4e50-8588-35fbee8d4df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:14.512796 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-3d35048c-e348-4e50-8588-35fbee8d4df3 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:14:14.513999 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3d35048c-e348-4e50-8588-35fbee8d4df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:14.571291 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1422/2854] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:14:14.581131 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:14.583053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:14.639502 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a49af-7259-492a-81c9-55d09b9d805d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:14.643172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7cd14-05c8-49c5-89f2-d820e600ed55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:14.643773 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7cd14-05c8-49c5-89f2-d820e600ed55 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:14.644551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7cd14-05c8-49c5-89f2-d820e600ed55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:14.644817 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7cd14-05c8-49c5-89f2-d820e600ed55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:14.653612 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7cd14-05c8-49c5-89f2-d820e600ed55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:14.655055 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7cd14-05c8-49c5-89f2-d820e600ed55 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:14.664833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1433/2855] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 10:14:14.925727 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35048c-e348-4e50-8588-35fbee8d4df3 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:14.927248 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-3d35048c-e348-4e50-8588-35fbee8d4df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:14.939195 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35048c-e348-4e50-8588-35fbee8d4df3 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:14.944355 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:14.944978 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:14.945286 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:14.946236 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] PATH_INFO: `/v3/projects/bc50ec3836b94cca8230d876191e360f` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:14.946841 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc50ec3836b94cca8230d876191e360f)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:14.948747 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:14.994689 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc50ec3836b94cca8230d876191e360f', 'request_id': 'req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:15.000658 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc50ec3836b94cca8230d876191e360f', 'request_id': 'req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:15.008849 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc50ec3836b94cca8230d876191e360f', 'request_id': 'req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:15.019690 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] The token cache is being invalidate because project bc50ec3836b94cca8230d876191e360f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:14:15.019984 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:15.020606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-15f0ce28-9d16-4ec6-9fd7-cbd26aec49a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:15.022062 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1423/2856] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:14:14 2022] DELETE /identity/v3/projects/bc50ec3836b94cca8230d876191e360f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:14:15.027009 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a460b-a6e2-486a-89d6-9566df55b06b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.027301 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a460b-a6e2-486a-89d6-9566df55b06b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.027580 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a460b-a6e2-486a-89d6-9566df55b06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.046583 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ae5a460b-a6e2-486a-89d6-9566df55b06b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:14:15.047822 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ae5a460b-a6e2-486a-89d6-9566df55b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:15.064881 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:14:15.109173 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1434/2857] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:14:15.169239 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33527c6-a253-4828-a31f-9cffc9e60941 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.169483 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33527c6-a253-4828-a31f-9cffc9e60941 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.169755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33527c6-a253-4828-a31f-9cffc9e60941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.189900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-a33527c6-a253-4828-a31f-9cffc9e60941 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:14:15.191717 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-a33527c6-a253-4828-a31f-9cffc9e60941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:15.199455 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1424/2858] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:14:15.203669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a460b-a6e2-486a-89d6-9566df55b06b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:15.213334 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a460b-a6e2-486a-89d6-9566df55b06b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:15.217847 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725da17f-2d00-4274-93bf-01f5b6e35c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:15.218438 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725da17f-2d00-4274-93bf-01f5b6e35c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.218718 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725da17f-2d00-4274-93bf-01f5b6e35c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.219017 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725da17f-2d00-4274-93bf-01f5b6e35c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.219669 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725da17f-2d00-4274-93bf-01f5b6e35c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:14:15.219971 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725da17f-2d00-4274-93bf-01f5b6e35c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:15.221549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725da17f-2d00-4274-93bf-01f5b6e35c39 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:15.232629 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1435/2859] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:14:15.236659 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33527c6-a253-4828-a31f-9cffc9e60941 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:15.246093 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33527c6-a253-4828-a31f-9cffc9e60941 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:15.250598 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43571dd-41b9-4d20-89f8-b1084335f6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:15.251212 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43571dd-41b9-4d20-89f8-b1084335f6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.251490 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43571dd-41b9-4d20-89f8-b1084335f6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.251761 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43571dd-41b9-4d20-89f8-b1084335f6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.252625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43571dd-41b9-4d20-89f8-b1084335f6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:15.255157 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43571dd-41b9-4d20-89f8-b1084335f6b7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:15.280889 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1425/2860] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:14:15 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:14:15.284911 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725da17f-2d00-4274-93bf-01f5b6e35c39 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:15.295030 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725da17f-2d00-4274-93bf-01f5b6e35c39 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:15.349299 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f9e40-533d-4b5b-956a-b952087c1b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:15.349920 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f9e40-533d-4b5b-956a-b952087c1b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.350202 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f9e40-533d-4b5b-956a-b952087c1b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.350451 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f9e40-533d-4b5b-956a-b952087c1b8a admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.351080 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f9e40-533d-4b5b-956a-b952087c1b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:15.352924 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f9e40-533d-4b5b-956a-b952087c1b8a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:15.372745 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1436/2861] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:14:15 2022] POST /identity/v3/users => generated 286 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:14:15.377477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43571dd-41b9-4d20-89f8-b1084335f6b7 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:15.388110 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43571dd-41b9-4d20-89f8-b1084335f6b7 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:15.392628 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb4b56-2e2e-4494-89bd-a67a5211bf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:15.393205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb4b56-2e2e-4494-89bd-a67a5211bf3f admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.393477 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb4b56-2e2e-4494-89bd-a67a5211bf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.393741 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb4b56-2e2e-4494-89bd-a67a5211bf3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.394361 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb4b56-2e2e-4494-89bd-a67a5211bf3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:15.395955 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb4b56-2e2e-4494-89bd-a67a5211bf3f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:15.405596 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1426/2862] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:14:15.409571 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f9e40-533d-4b5b-956a-b952087c1b8a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:15.419577 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f9e40-533d-4b5b-956a-b952087c1b8a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:15.424209 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65d621-5ebb-43e4-b195-109b0b5dbc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:15.424795 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65d621-5ebb-43e4-b195-109b0b5dbc59 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.425072 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65d621-5ebb-43e4-b195-109b0b5dbc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.425833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65d621-5ebb-43e4-b195-109b0b5dbc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.426465 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65d621-5ebb-43e4-b195-109b0b5dbc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:15.428074 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65d621-5ebb-43e4-b195-109b0b5dbc59 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:15.438399 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1437/2863] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:14:15.442900 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb4b56-2e2e-4494-89bd-a67a5211bf3f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:15.452555 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb4b56-2e2e-4494-89bd-a67a5211bf3f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:15.457075 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229beeb-3fa0-46d9-a5e6-809ee0c69981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:15.457704 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229beeb-3fa0-46d9-a5e6-809ee0c69981 admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.457975 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229beeb-3fa0-46d9-a5e6-809ee0c69981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.458243 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229beeb-3fa0-46d9-a5e6-809ee0c69981 admin admin] PATH_INFO: `/v3/projects/8403f819e5844bdeb7617fc60c671eca/users/0998826813db4f8ea6db0b0e478e095f/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.472489 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229beeb-3fa0-46d9-a5e6-809ee0c69981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8403f819e5844bdeb7617fc60c671eca, user_id=0998826813db4f8ea6db0b0e478e095f, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:15.475131 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229beeb-3fa0-46d9-a5e6-809ee0c69981 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:15.487514 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1427/2864] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:14:15 2022] PUT /identity/v3/projects/8403f819e5844bdeb7617fc60c671eca/users/0998826813db4f8ea6db0b0e478e095f/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:14:15.497724 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5095b-7328-4a1d-b9fe-addd20edb0c0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.498010 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5095b-7328-4a1d-b9fe-addd20edb0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.498284 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5095b-7328-4a1d-b9fe-addd20edb0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.526941 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-89a5095b-7328-4a1d-b9fe-addd20edb0c0 None None] MFA Rules not processed for user `0998826813db4f8ea6db0b0e478e095f`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:14:15.528180 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-89a5095b-7328-4a1d-b9fe-addd20edb0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:15.590676 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1438/2865] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:14:15.604085 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229beeb-3fa0-46d9-a5e6-809ee0c69981 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:15.605790 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-8229beeb-3fa0-46d9-a5e6-809ee0c69981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:15.627578 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:14:15.664755 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229beeb-3fa0-46d9-a5e6-809ee0c69981 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:15.668525 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a9d1c-04ac-43c6-89e1-f95cbca91518 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8f456f0d15b4d068cfcd6476563247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:15.669126 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a9d1c-04ac-43c6-89e1-f95cbca91518 service glance] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:15.669406 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a9d1c-04ac-43c6-89e1-f95cbca91518 service glance] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:15.669678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a9d1c-04ac-43c6-89e1-f95cbca91518 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:15.678623 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a9d1c-04ac-43c6-89e1-f95cbca91518 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:15.680104 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a9d1c-04ac-43c6-89e1-f95cbca91518 service glance] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:15.688810 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1428/2866] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:14:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:14:25.587845 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:14:25.684890 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=17 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:14:41.323939 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5095b-7328-4a1d-b9fe-addd20edb0c0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:41.383953 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5095b-7328-4a1d-b9fe-addd20edb0c0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:41.388367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:41.388979 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:41.389254 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:41.389523 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] PATH_INFO: `/v3/users/0998826813db4f8ea6db0b0e478e095f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:41.390619 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0998826813db4f8ea6db0b0e478e095f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:41.392551 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:41.438175 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0998826813db4f8ea6db0b0e478e095f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.446078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0998826813db4f8ea6db0b0e478e095f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.457448 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0998826813db4f8ea6db0b0e478e095f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.458098 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0998826813db4f8ea6db0b0e478e095f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.458693 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0998826813db4f8ea6db0b0e478e095f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.471262 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1439/2867] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:14:41 2022] DELETE /identity/v3/users/0998826813db4f8ea6db0b0e478e095f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:14:41.475862 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89d0c2-2e1c-45dc-9e73-7f933c0d522b None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:41.476387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89d0c2-2e1c-45dc-9e73-7f933c0d522b None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:41.476387 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89d0c2-2e1c-45dc-9e73-7f933c0d522b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:41.495267 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-dc89d0c2-2e1c-45dc-9e73-7f933c0d522b None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:14:41.496435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-dc89d0c2-2e1c-45dc-9e73-7f933c0d522b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:41.552437 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1429/2868] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:14:41.561936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:41.563358 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:41.619316 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a3eea-0b1e-4d54-a5c6-d8b043254d91 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:41.622910 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c3880-d281-4aa6-b5e1-8f03e367adb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:41.623530 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c3880-d281-4aa6-b5e1-8f03e367adb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:41.623810 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c3880-d281-4aa6-b5e1-8f03e367adb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:41.624078 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c3880-d281-4aa6-b5e1-8f03e367adb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:41.633727 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c3880-d281-4aa6-b5e1-8f03e367adb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:41.635172 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c3880-d281-4aa6-b5e1-8f03e367adb9 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:41.644423 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1440/2869] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:14:41.900812 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89d0c2-2e1c-45dc-9e73-7f933c0d522b None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:41.902242 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-dc89d0c2-2e1c-45dc-9e73-7f933c0d522b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:41.914604 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89d0c2-2e1c-45dc-9e73-7f933c0d522b None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:41.919449 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:41.920072 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:41.920348 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:41.920616 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] PATH_INFO: `/v3/projects/8403f819e5844bdeb7617fc60c671eca` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:41.921729 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=8403f819e5844bdeb7617fc60c671eca)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:41.923708 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:41.969046 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8403f819e5844bdeb7617fc60c671eca', 'request_id': 'req-b699ab31-b7c7-4203-81f5-5e4298951cab'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.975734 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8403f819e5844bdeb7617fc60c671eca', 'request_id': 'req-b699ab31-b7c7-4203-81f5-5e4298951cab'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.982880 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8403f819e5844bdeb7617fc60c671eca', 'request_id': 'req-b699ab31-b7c7-4203-81f5-5e4298951cab'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.993750 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] The token cache is being invalidate because project 8403f819e5844bdeb7617fc60c671eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:14:41.994029 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.994635 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-b699ab31-b7c7-4203-81f5-5e4298951cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:14:41.996114 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1430/2870] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:14:41 2022] DELETE /identity/v3/projects/8403f819e5844bdeb7617fc60c671eca => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:14:42.000955 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5040c7c-fa62-4de8-9ec5-6c6898b46a9a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.001239 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5040c7c-fa62-4de8-9ec5-6c6898b46a9a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.001513 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5040c7c-fa62-4de8-9ec5-6c6898b46a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.020717 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c5040c7c-fa62-4de8-9ec5-6c6898b46a9a None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:14:42.021902 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c5040c7c-fa62-4de8-9ec5-6c6898b46a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:42.039317 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:14:42.083686 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1441/2871] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:14:42.095373 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58187a3c-d762-4dfc-9a8f-cb6b95c8c0d1 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.095789 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58187a3c-d762-4dfc-9a8f-cb6b95c8c0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.096064 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58187a3c-d762-4dfc-9a8f-cb6b95c8c0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.116251 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-58187a3c-d762-4dfc-9a8f-cb6b95c8c0d1 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:14:42.117435 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-58187a3c-d762-4dfc-9a8f-cb6b95c8c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:42.125165 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1431/2872] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 10:14:42.129723 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5040c7c-fa62-4de8-9ec5-6c6898b46a9a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:42.139174 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5040c7c-fa62-4de8-9ec5-6c6898b46a9a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:42.143872 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0930d-e876-43f0-bd0a-f2411e377c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:42.144999 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0930d-e876-43f0-bd0a-f2411e377c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.145276 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0930d-e876-43f0-bd0a-f2411e377c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.145541 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0930d-e876-43f0-bd0a-f2411e377c67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.146179 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0930d-e876-43f0-bd0a-f2411e377c67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:14:42.146476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0930d-e876-43f0-bd0a-f2411e377c67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:42.148367 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0930d-e876-43f0-bd0a-f2411e377c67 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:42.159560 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1442/2873] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 10 10:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:14:42.164037 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58187a3c-d762-4dfc-9a8f-cb6b95c8c0d1 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:42.172824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58187a3c-d762-4dfc-9a8f-cb6b95c8c0d1 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:42.177350 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef606d3e-eacd-4ca9-bc3d-7349b4ad3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:42.177930 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef606d3e-eacd-4ca9-bc3d-7349b4ad3741 admin admin] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.178205 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef606d3e-eacd-4ca9-bc3d-7349b4ad3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.178471 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef606d3e-eacd-4ca9-bc3d-7349b4ad3741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.179107 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef606d3e-eacd-4ca9-bc3d-7349b4ad3741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:42.180959 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef606d3e-eacd-4ca9-bc3d-7349b4ad3741 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:42.208342 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1432/2874] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 10 10:14:42 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:14:42.212511 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0930d-e876-43f0-bd0a-f2411e377c67 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:42.229236 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0930d-e876-43f0-bd0a-f2411e377c67 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:42.282746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f851d-b279-40a2-be9e-46aefc23077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:42.283370 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f851d-b279-40a2-be9e-46aefc23077c admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.283643 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f851d-b279-40a2-be9e-46aefc23077c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.283908 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f851d-b279-40a2-be9e-46aefc23077c admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.284507 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f851d-b279-40a2-be9e-46aefc23077c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:42.287063 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f851d-b279-40a2-be9e-46aefc23077c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:42.307933 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1443/2875] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 10 10:14:42 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:14:42.312081 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef606d3e-eacd-4ca9-bc3d-7349b4ad3741 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:42.322264 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef606d3e-eacd-4ca9-bc3d-7349b4ad3741 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:42.326776 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec40bb6-ae9c-4b17-89ac-2719b6e1b616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:42.327386 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec40bb6-ae9c-4b17-89ac-2719b6e1b616 admin admin] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.327661 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec40bb6-ae9c-4b17-89ac-2719b6e1b616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.327928 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec40bb6-ae9c-4b17-89ac-2719b6e1b616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.328553 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec40bb6-ae9c-4b17-89ac-2719b6e1b616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:42.330112 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec40bb6-ae9c-4b17-89ac-2719b6e1b616 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:42.340181 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1433/2876] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:14:42.344069 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f851d-b279-40a2-be9e-46aefc23077c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:42.354415 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f851d-b279-40a2-be9e-46aefc23077c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:42.359188 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d0442-5a99-442f-86e2-45336d195cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:42.359769 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d0442-5a99-442f-86e2-45336d195cea admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.360043 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d0442-5a99-442f-86e2-45336d195cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.360311 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d0442-5a99-442f-86e2-45336d195cea admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.360936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d0442-5a99-442f-86e2-45336d195cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:42.362515 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d0442-5a99-442f-86e2-45336d195cea admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:42.372818 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1444/2877] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 10 10:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 10:14:42.376799 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec40bb6-ae9c-4b17-89ac-2719b6e1b616 admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:42.386681 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec40bb6-ae9c-4b17-89ac-2719b6e1b616 admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:42.391207 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702132e9-e65a-4f26-809d-fede6f2b7d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:42.391831 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702132e9-e65a-4f26-809d-fede6f2b7d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.392106 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702132e9-e65a-4f26-809d-fede6f2b7d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.392375 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702132e9-e65a-4f26-809d-fede6f2b7d4f admin admin] PATH_INFO: `/v3/projects/033236cd692e4f929bec9125caa5d715/users/0d86e0e32f4a499fa7ca250534489fdd/roles/7694dcfc7d6547abb6565cb9e726eacd` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.411606 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702132e9-e65a-4f26-809d-fede6f2b7d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=033236cd692e4f929bec9125caa5d715, user_id=0d86e0e32f4a499fa7ca250534489fdd, role_id=7694dcfc7d6547abb6565cb9e726eacd)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:42.412253 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702132e9-e65a-4f26-809d-fede6f2b7d4f admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:42.424625 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1434/2878] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 10 10:14:42 2022] PUT /identity/v3/projects/033236cd692e4f929bec9125caa5d715/users/0d86e0e32f4a499fa7ca250534489fdd/roles/7694dcfc7d6547abb6565cb9e726eacd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:14:42.434783 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e1fa3-d67a-4472-977c-4547e2a74444 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.435091 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e1fa3-d67a-4472-977c-4547e2a74444 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.435363 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e1fa3-d67a-4472-977c-4547e2a74444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.462549 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cd6e1fa3-d67a-4472-977c-4547e2a74444 None None] MFA Rules not processed for user `0d86e0e32f4a499fa7ca250534489fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:14:42.463764 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cd6e1fa3-d67a-4472-977c-4547e2a74444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:42.527833 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1445/2879] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:14:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:14:42.539393 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702132e9-e65a-4f26-809d-fede6f2b7d4f admin admin] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:14:42.541086 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-702132e9-e65a-4f26-809d-fede6f2b7d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:14:42.561760 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:14:42.598330 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702132e9-e65a-4f26-809d-fede6f2b7d4f admin admin] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:14:42.601809 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55766f-954e-47d1-ba53-dbf3be1727db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1e3ba8d0c7d4f8190cd0b5b057f5638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:14:42.602394 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55766f-954e-47d1-ba53-dbf3be1727db service cinder] REQUEST_METHOD: `GET` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:14:42.602664 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55766f-954e-47d1-ba53-dbf3be1727db service cinder] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:14:42.602926 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55766f-954e-47d1-ba53-dbf3be1727db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:14:42.610782 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55766f-954e-47d1-ba53-dbf3be1727db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:14:42.612208 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55766f-954e-47d1-ba53-dbf3be1727db service cinder] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:14:42.620577 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1435/2880] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:14:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:14:52.524936 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:14:52.616773 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=17 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:15:15.454158 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e1fa3-d67a-4472-977c-4547e2a74444 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:15:15.514760 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e1fa3-d67a-4472-977c-4547e2a74444 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:15:15.519313 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:15:15.519938 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:15:15.520220 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:15:15.520495 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] PATH_INFO: `/v3/users/0d86e0e32f4a499fa7ca250534489fdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:15:15.521662 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d86e0e32f4a499fa7ca250534489fdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:15:15.523666 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:15:15.569342 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d86e0e32f4a499fa7ca250534489fdd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:15.576134 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d86e0e32f4a499fa7ca250534489fdd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:15.588053 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d86e0e32f4a499fa7ca250534489fdd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:15.588737 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d86e0e32f4a499fa7ca250534489fdd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:15.589352 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d86e0e32f4a499fa7ca250534489fdd'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:15.602564 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1446/2881] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 10 10:15:15 2022] DELETE /identity/v3/users/0d86e0e32f4a499fa7ca250534489fdd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:15:15.607027 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b38e9-e8cc-4e44-b336-ef3efe9a7708 None None] REQUEST_METHOD: `POST` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:15:15.607543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b38e9-e8cc-4e44-b336-ef3efe9a7708 None None] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:15:15.607543 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b38e9-e8cc-4e44-b336-ef3efe9a7708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:15:15.626080 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.auth.core [None req-633b38e9-e8cc-4e44-b336-ef3efe9a7708 None None] MFA Rules not processed for user `2ff7daf364734feb8aab110b9b2ef63b`. Rule list: `[]` (Enabled: `True`). {{(pid=80246) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:15:15.627279 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-633b38e9-e8cc-4e44-b336-ef3efe9a7708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:15:15.682858 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1436/2882] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 10 10:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 10:15:15.692351 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:15:15.693746 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:15:15.749733 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369b72d-95f6-4b58-9349-ac993bb01c93 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:15:15.753604 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787cb6-6b3f-4e78-82d4-866f15d5bec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab9b7576a3834862af6abb7e0416f158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:15:15.754195 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787cb6-6b3f-4e78-82d4-866f15d5bec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:15:15.754469 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787cb6-6b3f-4e78-82d4-866f15d5bec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:15:15.754738 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787cb6-6b3f-4e78-82d4-866f15d5bec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:15:15.763449 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95787cb6-6b3f-4e78-82d4-866f15d5bec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:15:15.764869 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95787cb6-6b3f-4e78-82d4-866f15d5bec0 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:15:15.774554 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1447/2883] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:15:16.042198 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b38e9-e8cc-4e44-b336-ef3efe9a7708 None None] Authenticating user token {{(pid=80246) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:15:16.043753 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.fernet_utils [None req-633b38e9-e8cc-4e44-b336-ef3efe9a7708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80246) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:15:16.056289 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b38e9-e8cc-4e44-b336-ef3efe9a7708 None None] Validating token access rules against request {{(pid=80246) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:15:16.063824 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff7daf364734feb8aab110b9b2ef63b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cee49555c8143ed868a4be75b65ed07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80246) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:15:16.064451 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:15:16.064724 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:15:16.064992 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] PATH_INFO: `/v3/projects/033236cd692e4f929bec9125caa5d715` {{(pid=80246) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:15:16.066821 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=033236cd692e4f929bec9125caa5d715)` {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:15:16.069384 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] RBAC: Authorization granted {{(pid=80246) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:15:16.115962 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033236cd692e4f929bec9125caa5d715', 'request_id': 'req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:16.121940 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033236cd692e4f929bec9125caa5d715', 'request_id': 'req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:16.128678 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033236cd692e4f929bec9125caa5d715', 'request_id': 'req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7'} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:16.139187 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] The token cache is being invalidate because project 033236cd692e4f929bec9125caa5d715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80246) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 10:15:16.139468 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:16.140071 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG keystone.notifications [None req-90eb32b9-80c8-4518-96ae-f4034d7f8dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80246) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 10:15:16.141535 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: [pid: 80246|app: 0|req: 1437/2884] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 10 10:15:16 2022] DELETE /identity/v3/projects/033236cd692e4f929bec9125caa5d715 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:15:25.770102 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:15:26.136285 dv-cinder-850836-5-iscsi devstack@keystone.service[80246]: DEBUG dbcounter [-] [80246] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80246) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:16:21.005269 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787cb6-6b3f-4e78-82d4-866f15d5bec0 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:16:21.006655 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-95787cb6-6b3f-4e78-82d4-866f15d5bec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:16:21.073846 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787cb6-6b3f-4e78-82d4-866f15d5bec0 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:16:21.131476 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6e920-164a-4abf-a592-aade0cde7897 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3d7b3cb498a4da18e203456e09b59e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8afc1639a0c4c6c86a1d86d5c254513', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:16:21.132079 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6e920-164a-4abf-a592-aade0cde7897 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:16:21.132356 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6e920-164a-4abf-a592-aade0cde7897 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:16:21.132627 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6e920-164a-4abf-a592-aade0cde7897 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:16:21.134381 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b2f6e920-164a-4abf-a592-aade0cde7897 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:16:21.145427 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6e920-164a-4abf-a592-aade0cde7897 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:16:21.147003 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6e920-164a-4abf-a592-aade0cde7897 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:16:21.157108 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1448/2885] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 10 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 10:16:31.152926 dv-cinder-850836-5-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=58 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}